NEMIS

CONCEPT

OF

OPERATIONS

(OPERATIONS AND MAINTENANCE)

![](media/image1.wmf){width="3.448611111111111in"
height="3.448611111111111in"}

FEMA

Information Technology Services Directorate

Operations Division

May 15, 1998

TABLE OF CONTENTS

1\. Introduction [1](#introduction)

1.1 Scope [1](#scope)

1.2 Objectives [1](#objectives)

1.3 Referenced documents [1](#referenced-documents)

2\. NEMIS Overview [3](#nemis-overview)

2.1 NEMIS Schedule [5](#nemis-schedule)

2.2 NEMIS Functions [5](#nemis-functions)

2.2.1 Oracle Relational Data Base Management System (RDBMS)
[5](#oracle-relational-data-base-management-system-rdbms)

2.2.2 ViewStar Workflow Management [6](#viewstar-workflow-management)

2.2.3 NEMIS Intranet WEB Server [6](#nemis-intranet-web-server)

2.2.4 Microsoft NT Server [7](#microsoft-nt-server)

2.2.5 Microsoft Exchange Server [7](#microsoft-exchange-server)

2.2.6 Emergency Information System (EIS) InfoBook and MapInfo
[7](#emergency-information-system-eis-infobook-and-mapinfo)

2.2.7 Oracle Discoverer [8](#oracle-discoverer)

2.2.8 Configuration Control Management
[8](#configuration-control-management)

2.2.9 Local Area Network/Wide Area Network (LAN/WAN)
[8](#local-area-networkwide-area-network-lanwan)

2.2.10 Client Workstation Support [8](#client-workstation-support)

2.2.11 NEMIS Security [8](#nemis-security)

3\. General Concept of Operations [10](#general-concept-of-operations)

3.1 Approach [10](#approach)

3.1.1 Tier One Support [12](#tier-one-support)

3.1.1.1 Overview of NEMIS Functions Supported
[13](#overview-of-nemis-functions-supported)

3.1.1.2 Skill Set Required [13](#skill-set-required)

3.1.1.3 Locations [14](#locations)

3.1.2 Tier Two Support [14](#tier-two-support)

3.1.2.1 Overview of NEMIS Functions Supported
[14](#overview-of-nemis-functions-supported-1)

3.1.2.2 Skill Set Required [15](#skill-set-required-1)

3.1.2.3 Locations [15](#locations-1)

3.1.3 Tier Three, Level 1 Support [16](#tier-three-level-1-support)

3.1.3.1 Overview of NEMIS Functions Supported
[16](#overview-of-nemis-functions-supported-2)

3.1.3.2 Skill Sets Required [17](#skill-sets-required)

3.1.3.3 Tier Three, Level 1 Locations
[17](#tier-three-level-1-locations)

3.1.4 Tier Three, Level 2 Support [17](#tier-three-level-2-support)

3.1.4.1 Overview of NEMIS Functions Supported
[18](#overview-of-nemis-functions-supported-3)

3.1.4.2 Skill Sets Required [19](#skill-sets-required-1)

3.1.4.3 Tier Three, Level Two Locations
[20](#tier-three-level-two-locations)

3.1.5 Tier Three, Level 3 Support [20](#tier-three-level-3-support)

3.1.5.1 Overview of NEMIS Functions Supported
[21](#overview-of-nemis-functions-supported-4)

3.1.5.2 Skill Sets Required [21](#skill-sets-required-2)

3.1.5.3 Tier Three, Level 3 Locations
[22](#tier-three-level-3-locations)

3.1.6 Mobile/Remote Support Teams [22](#mobileremote-support-teams)

3.1.6.1 Overview of NEMIS Functions Supported
[22](#overview-of-nemis-functions-supported-5)

3.1.6.2 Skill Set Required [23](#skill-set-required-2)

3.1.6.3 Locations [23](#locations-2)

3.2 Localized Administrative Functions (Tier 1 & 2)
[23](#localized-administrative-functions-tier-1-2)

3.2.1 Localized Oracle RDBMS Database Administrative Functions (Tier 2)
[23](#localized-oracle-rdbms-database-administrative-functions-tier-2)

3.2.2 Localized ViewStar Administrative Functions (Tier 2)
[24](#localized-viewstar-administrative-functions-tier-2)

3.2.3 Localized Server/LAN Administration (Tier 2)
[24](#localized-serverlan-administration-tier-2)

3.2.4 Localized WAN Support Functions (Tier 2)
[26](#localized-wan-support-functions-tier-2)

3.2.5 Localized Help Desk Support (Tier 1 & 2)
[26](#localized-help-desk-support-tier-1-2)

3.3 Centralized Administrative Functions (Tier 3, Level 1 & 2)
[27](#centralized-administrative-functions-tier-3-level-1-2)

3.3.1 Centralized Enterprise Management Approach
[27](#centralized-enterprise-management-approach)

3.3.1.1 BMC Patrol [27](#bmc-patrol)

3.3.1.2 Tivoli User Administration [28](#tivoli-user-administration)

3.3.1.3 MS System Management Server (SMS)
[28](#ms-system-management-server-sms)

3.3.2 Overall Centralized NEMIS Management Functions (Tier 3, Level 2
and Senior Management)
[29](#overall-centralized-nemis-management-functions-tier-3-level-2-and-senior-management)

3.3.3 Centralized Oracle Database Administrative Functions (Tier 3,
Level 2)
[30](#centralized-oracle-database-administrative-functions-tier-3-level-2)

3.3.4 Centralized ViewStar Administrative Functions (Tier Three, Level
2)
[32](#centralized-viewstar-administrative-functions-tier-three-level-2)

3.3.5 Centralized WAN Administrative Functions (Tier 3, Level 2)
[32](#centralized-wan-administrative-functions-tier-3-level-2)

3.3.6 Centralized Server/LAN Administrative Functions (Tier 3, Level 2)
[33](#centralized-serverlan-administrative-functions-tier-3-level-2)

3.3.7 Centralized Configuration Management Support Functions (Tier 3,
Level 2)
[34](#centralized-configuration-management-support-functions-tier-3-level-2)

3.3.8 Centralized Security Support Functions (Tier 3, Level 2)
[35](#centralized-security-support-functions-tier-3-level-2)

3.3.9 Centralized WWW Administrative Functions (Tier 3, Level 2)
[35](#centralized-www-administrative-functions-tier-3-level-2)

3.3.10 National Helpdesk Support (Tier 3, Level 1)
[35](#national-helpdesk-support-tier-3-level-1)

4\. Skill Set Matrix [37](#skill-set-matrix)

5\. Site Staffing Requirements [39](#site-staffing-requirements)

5.1 Centralized Staffing Requirement
[39](#centralized-staffing-requirement)

5.2 Localized Fixed Facility Staffing Requirements
[40](#localized-fixed-facility-staffing-requirements)

5.2.1 Mount Weather Staffing and Skill Set Requirements
[41](#mount-weather-staffing-and-skill-set-requirements)

5.2.2 Denton Staffing Requirements [42](#denton-staffing-requirements)

5.2.3 Hyattsville NPSC/NTC Staffing Requirements
[43](#hyattsville-npscntc-staffing-requirements)

5.2.4 FEMA HQ Staffing Requirements [43](#fema-hq-staffing-requirements)

5.3 Regional Staffing Requirements [44](#regional-staffing-requirements)

6\. Maintenance [48](#maintenance)

7\. Assumptions and Constraints [49](#assumptions-and-constraints)

APENDIXES

a\. Proposed BMC Architecture for NEMIS
a-[1](#proposed-bmc-architecture-for-nemis)

b\. Proposed Tivoli Architecture for NEMIS
b-[1](#proposed-tivoli-architecture-for-nemis)

c\. Proposed SMS Architecture for NEMIS
c-[1](#proposed-sms-architecture-for-nemis)

LIST OF FIGURES

Figure 3‑1 Tier Three Enterprise Support Levels
[11](#__RefHeading___Toc419855762)

Figure 3‑2 Overview of Support Request Flow Between Tiers
[12](#__RefHeading___Toc419855763)

Figure 3‑1 Localized Support - Tier One Support Structure
[13](#__RefHeading___Toc419855764)

Figure 3‑1 Localized Support - Tier Two Support Structure
[14](#__RefHeading___Toc419855765)

Figure 3‑1 Centralized Support - Tier Three, Level 1 Support Structure
[16](#__RefHeading___Toc419855766)

Figure 3‑1 Centralized Support - Tier Three, Level 2 Support Structure
[18](#__RefHeading___Toc419855767)

Figure 3‑1 Centralized Engineering/Development - Tier Three, Level 3
Structure [20](#__RefHeading___Toc419855768)

Figure b‑1 -- Single TMR (Conceptual View)
[2](#__RefHeading___Toc419855769)

Figure b‑2 -- Sample Tivoli Policy Region
[4](#__RefHeading___Toc419855770)

Figure b‑3 Tivoli Users Icon [4](#__RefHeading___Toc419855771)

Figure b‑4 Tivoli Group Icon [4](#__RefHeading___Toc419855772)

Figure b‑5 Sample Admin Icon [7](#__RefHeading___Toc419855773)

Figure b‑6 Sample Senior Admin Icon [7](#__RefHeading___Toc419855774)

LIST OF TABLES

Table 4‑1 Centralized NEMIS Skill Set Requirement Matrix
[38](#__RefHeading___Toc419855775)

Table 4‑2 Localized NEMIS Skill Set Requirement Matrix
[38](#__RefHeading___Toc419855776)

Table 5.1‑1 Estimated NEMIS Centralized Staffing Requirements
[40](#__RefHeading___Toc419855777)

Table 5.2‑1 Estimated NEMIS Localized Fixed Site Staffing Requirements
[41](#__RefHeading___Toc419855778)

Table 5.2.1‑1 Centralized Enterprise Staffing Requirements (Mount
Weather) [42](#__RefHeading___Toc419855779)

Table 5.2.1‑2 MWEAC Site Staffing Requirements
[42](#__RefHeading___Toc419855780)

Table 5.2.1‑3 Virginia NPSC/NTC Staffing Requirements
[42](#__RefHeading___Toc419855781)

Table 5.2.2‑1 Denton NTC Staffing Requirements
[42](#__RefHeading___Toc419855782)

Table 5.2.2‑2 Denton NPSC Staffing Requirements
[43](#__RefHeading___Toc419855783)

Table 5.2.3‑1 Hyattsville NPSC/NTC Staffing Requirements
[43](#__RefHeading___Toc419855784)

Table 5.2.4‑1 HQ Site Staffing Requirements
[43](#__RefHeading___Toc419855785)

Table 5.3‑1 Region I Staffing Requirements
[44](#__RefHeading___Toc419855786)

Table 5.3‑2 Region II Staffing Requirements
[44](#__RefHeading___Toc419855787)

Table 5.3‑3 Region III Staffing Requirements
[45](#__RefHeading___Toc419855788)

Table 5.3‑4 Region IV Staffing Requirements
[45](#__RefHeading___Toc419855789)

Table 5.3‑5 Region V Staffing Requirements
[45](#__RefHeading___Toc419855790)

Table 5.3‑6 Region VI Staffing Requirements
[46](#__RefHeading___Toc419855791)

Table 5.3‑7 Region VII Staffing Requirements
[46](#__RefHeading___Toc419855792)

Table 5.3‑8 Region VIII Staffing Requirements
[46](#__RefHeading___Toc419855793)

Table 5.3‑9 Region IX Staffing Requirements
[46](#__RefHeading___Toc419855794)

Table 5.3‑10 Region X Staffing Requirements
[47](#__RefHeading___Toc419855795)

# Introduction

The Federal Emergency Management Agency's (FEMA) primary charter is to
provide assistance to individuals, businesses, and state, and local
governments during a time of disaster. This assistance takes many forms.
Disaster mitigation, preparation, response, and recovery are all aspects
of disaster assistance. In the past, FEMA automated a portion of this
support, but for the most part, these tasks were satisfied via manual
preparation. The automated support was developed in the early 1980s and
has been overtaken by new technologies and functional requirements. For
this reason FEMA began development of the National Emergency Management
Information System (NEMIS).

While NEMIS introduces a powerful method for providing automated support
to FEMA employees in their day-to-day disaster assistance activities, it
also generates new requirements for operations and maintenance (O&M).
FEMA must analyze the O&M requirements and plan for the implementation
of the NEMIS system and its associated hardware and software life cycle
support.

## Scope

This document addresses FEMA's requirement for operating and maintaining
the NEMIS system. It briefly discusses the NEMIS system and its
components, and all associated systems that impact NEMIS operation. It
is provided as a draft for developing a NEMIS concept of operations.

## Objectives

This document was developed to provide an overview of the NEMIS system,
identify its key O&M requirements, quantify those requirements in the
form of skill levels and positions, and provide a general approach for
operating and maintaining the system into the future.

Section 2 provides an overview of the NEMIS system. Section 3 merges the
system and administrative requirements into a general concept of
operations for NEMIS. Section 4 identifies the administrative support
requirements of the proposed system broken down by skill sets. Sections
5 and 6 briefly address maintenance and training. Section identifies the
assumptions and constraints which impact the O&M functional
requirements. Appendix A, Appendix B, and Appendix C cover the proposed
implementation of the O&M NEMIS administrative tools (i.e., BMC Patrol,
Tivoli, and MS System Management Server).

## Referenced documents

-   Final NEMIS Software Product Alternatives Analysis, Deliverable 9,
    June 27, 1996 through Revision B. Prepared for the Federal Emergency
    Management Agency, Information Technology Services Directorate,
    Applications Development Division, Major Programs Branch, through
    the General Services Administration, National Capital Region,
    Contract GS11K94BJD0001, Task Number N1N586003, Work Request Number
    FEMA 0001.

-   Draft NEMIS System/Subsystem Specification (SSS), Deliverable 7,
    July 1, 1996 through Revision D. Prepared for the Federal Emergency
    Management Agency, Information Technology Services Directorate,
    Applications Development Division, Major Programs Branch, through
    the General Services Administration, National Capital Region,
    Contract GS11K94BJD0001, Task Number N1N586003, Work Definition No.
    FEMA 0002.

# NEMIS Overview

NEMIS is a FEMA-wide system of hardware, software, telecommunications,
and applications that provides a new technology base to FEMA and its
partners to carry out the emergency management mission. NEMIS integrates
and automates tools to support operations for:

-   Incident Activities;

-   Preliminary Damage Assessments (PDA);

-   Declaration Activities;

-   External Interfaces to Other FEMA Systems and Other Government
    Systems;

-   Public Information;

-   Donations;

-   Application Processing;

```{=html}
<!-- -->
```
-   Human Services (HS);

-   Infrastructure Services (IS);

-   Mitigation Services (MT);

-   Emergency Coordination (EC);

```{=html}
<!-- -->
```
-   Inspections;

-   Helpline;

-   Program Planning & Budgeting;

-   Functional Data Administration;

-   Emergency Support (ES);

-   NEMIS-Wide Functions for Access to the FEMA Library, Imaging, Global
    Postings, Management Reporting. Action Tracking, and Online Help.

    This section provides a very high level description of the NEMIS
    system and subsystems. Under the Anteon development contract, the
    FEMA IT Programs Management Group (PMG) produced the NEMIS System
    Subsystem Design Document (SSDD). Appendix A of this document is an
    abridged version of the SSDD system description section. It provides
    a more detailed description of the NEMIS system and how it provides
    operational support for the functions listed above.

    NEMIS has expandability and hooks to include other FEMA operations
    as desired. NEMIS enables FEMA to use information as a strategic
    resource to provide effective and timely response, recovery,
    mitigation, and services. NEMIS provides managers with access to the
    data and analytical tools necessary for making effective plans and
    decisions. In addition to providing automated support for a full
    range to integrated emergency management processes, NEMIS is a
    comprehensive effort to interface with such other systems as:

-   Integrated Financial Management Information System (IFMIS);

-   Logistics Information Management System (LIMS) databases;

-   Automated Deployment Database (ADD);

-   National Emergency Coordination Center (NECC);

-   Personnel Computer Time and Attendance Remote Entry System
    (PC-TARE);

-   US Army Corps of Engineers system;

-   Natural Resource Conservation Service (NRSC) system;

-   Individual Family Grant (IFG) system;

-   Farm Service Agency (FSA) system;

-   Small Business Administration (SBA); and

-   Other agencies' systems.

    The NEMIS enterprise database is a collection of distributed
    disaster data and workflow databases that are linked together to
    permit the comprehensive retrieval of information across the entire
    FEMA enterprise. Common data formats and naming conventions allow
    existing and future applications to share and exchange data. NEMIS
    enables applications to share and exchange data, as they have been
    unable to do in the past. Information exchange among programs and
    organizational elements will facilitate coordinated FEMA/state
    emergency management.

    In addition, NEMIS is a valuable tool for increasing the partnership
    between FEMA and the states. This helps to coordinate the
    partnership with states by enabling the sharing of data and
    providing states with the same automated processes used by FEMA.
    NEMIS provides automated support for joint FEMA/state critical
    functions such as:

-   Managing infrastructure projects and grants;

-   Providing individual and family grants; and

-   Conducting PDAs.

Likewise, the state/FEMA partnership is crucial to all phases of NEMIS
development, from requirement definition, to testing and operations. To
ensure that NEMIS provides support integral to state operations,
National Emergency Management Association (NEMA) and state
representatives were involved with the NEMIS task forces. The NEMIS task
forces were responsible for defining the processes that NEMIS automated.

In addition to the states, FEMA works in close partnership with several
federal agencies that provide disaster-related services. NEMIS will
automate aspects of these relationships, such as the process of issuing
and tracking mission assignments, providing reimbursement for transient
accommodations, making SBA loan determinations, and others. In addition,
FEMA provides and receives information from a number of other federal
agencies (OFAs). NEMIS is planned to interface with several OFA systems
to replace current manual or ad hoc transmission of data. More
coordinated exchange of information reduces duplication of effort in
providing disaster assistance which results in better customer service
with a coordinated federal effort.

Again, Appendix A of this document provides a detailed description of
the NEMIS system and it's associated subsystems. Please refer to
Appendix A for a brief description of technical hardware, software, and
architecture functionality.

## NEMIS Schedule

The current NEMIS development schedule is depicted in Figure 2.1-1. Note
that the NEMIS development and implementation schedule is subject to
change.

![](media/image2.png){width="5.916666666666667in" height="4.3625in"}

Figure 2.1-1, NEMIS Development Schedule

## NEMIS Functions

The sections below identify NEMIS specific activities that O&M support.
These functions are NEMIS specific and may be a superset or subset of
current FEMA O&M functions.

### Oracle Relational Data Base Management System (RDBMS)

NEMIS uses Oracle at all sites for access control (username/password),
data manipulation, reporting, and data replication. However, while
Oracle is used at all sites, the support provided by a NEMIS/Oracle
system is dependent upon the function provided by that site. For
example, FEMA's National Processing Centers (NPSC) take request for
financial housing assistance for disaster victims. Therefore, NEMIS
disaster housing support is only provided at a NPSC site. A Disaster
Field Office (DFO) processes requests from state and local governments
to restore the disaster area infrastructure (IS). The FEMA region
supporting the disaster also provides this type of assistance.
Therefore, both the DFO and the Region NEMIS systems are equipped to
support IS.

This type of focused access provides the best environment for FEMA
employees, local governments, and disaster victims. However, maintenance
of this environment requires a large amount of administrative support
and a specialized understanding of where data resides, how it is
replicated, backed up, and recovered. NEMIS users access Oracle via
network SQL calls riding on Microsoft's TCP/IP protocol stack.

### ViewStar Workflow Management

NEMIS will automate many FEMA functions. This includes automating some
of the workflow through the FEMA work environment using a software
product called ViewStar. In the ViewStar environment, a NEMIS user
creates application folders that ViewStar forwards through the FEMA work
environment via a set of predefined queues coded into the NEMIS system.
The NEMIS business rules were derived from the original FEMA business
process analysis performed by the NEMIS developer. For example, FEMA
creates a disaster relief request when a disaster victim contacts FEMA
for disaster support. This request is automatically passed through the
NEMIS system by ViewStar queues, processing agents and business rule
analysis until all individuals responsible for authorizing the support
have reviewed the request and provide their input.

ViewStar uses Oracle's database for storage and tracking of application
folders and their associated documents. It also uses a flat file system
for queuing workpackets on the local NetWare file server. Workpackets
are processed and passed between the Oracle and file system by ViewStar
processing agents on the Microsoft (MS) NT servers.

The O&M requirements for maintaining the ViewStar environment are equal
to those imposed by the Oracle system. While the systems and
configurations remain equivalent from site to site, the data and
processing requirements are not consistent. ViewStar imposes the same
requirements for administrative support and specialized understanding of
where data resides, how it is replicated, backed up, and recovered.

### NEMIS Intranet WEB Server

NEMIS currently uses Oracle's WEB server for WWW based access to
specific NEMIS information. Authorized users with WEB browsers can
access NEMIS databases on the Oracle servers via TCP/IP, SQL, and HTML
protocols. This type of access, commonly referred to as "thin client"
access, may become more prevalent in future releases of the NEMIS
system. O&M requirements of these systems is directly dependent upon the
nature of the data, the size and complexity of the WEB services
provided. The more pages, links, and database ties a WEB site has, the
more O&M is required to maintain it. The NEMIS Intranet server will
maintain database links directly to the NEMIS Oracle data and therefore
require the "WEB Master" to have a working knowledge of the WEB server,
its specific ties to Oracle using PL/SQL routines, and the NEMIS Oracle
database environment.

### Microsoft NT Server

NT is the basis for all NEMIS server functions. However, access to these
functions are embedded in network database, workflow, and World Wide Web
(WWW) applications. Users will not be required to directly login to the
NEMIS NT server. While this may slightly ease O&M functions, O&M is
still required.

NT introduces a completely new network operating system (NOS)
environment into FEMA. Network Administrators must be trained in the new
environment and be conscious of the special configuration requirements
that Oracle and ViewStar impose on the environment. NT's version of
"enterprise networking" is accomplished via Domains. Implementing and
maintaining NT domains will be an integral part of the O&M enterprise
management of the NEMIS NT server environment. Server processes for
backing up the NT server must also incorporate the NEMIS support
applications (i.e., Oracle and ViewStar). Administrators must be
knowledgeable of the local area network (LAN) environment and must
possess skills for maintaining TCP/IP and SQL protocols as well as
FEMA's traditional NetWare IPX/SPX environment. NT represents a whole
new method for monitoring and maintaining the LAN and server
environments.

### Microsoft Exchange Server

MS Exchange server will be implemented to allow ViewStar to communicate
workflow application folders between sites and to enable communication
with field inspection palm pads. Like NT, MS Exchange is a foreign
system to FEMA's cc:Mail Administrators. When properly configured, MS
Exchange receives messages from clients and automatically and
intelligently forwards them to their recipients. cc:Mail uses a database
store-and-forward approach that requires external e-mail routers to pass
mail from site to site with limited intelligence. In the NEMIS
environment, O&M of MS Exchange will be limited to maintenance of the
ViewStar interface and for inspection services communications.

### Emergency Information System (EIS) InfoBook and MapInfo

EIS is a commercial off the shelf (COTS) application that has been
specifically designed to provide automated command and control functions
for emergency managers. InfoBook is a front-end for EIS that presents
NEMIS emergency data in the format of a three ring binder. The software
looks and works like a regular three-ring binder. Users access emergency
coordination information like maps, data, models, and communications by
using the mouse to point to the kind of information required. EIS
InfoBook comes in several editions, each with its own set of tabs and
capabilities selected to meet the needs of individual users. FEMA's
version of the software has been tailored to NEMIS requirements and has
been integrated with MapInfo Geographical Information System (GIS).
MapInfo provides NEMIS users with the capability to easily display
graphical emergency data in a geographical format. The emergency data
input to MapInfo will be extracted directly out of the NEMIS disaster
databases on the NT Oracle server. O&M's role for maintaining the
MapInfo is limited to providing general user support and loading new
MapInfo data sets and maps after they have been verified by the PMG data
support team.

### Oracle Discoverer

Specific NEMIS users have requirements to develop ad hoc queries against
NEMIS data. These users will use Oracle's Discoverer query software.
Users and support teams must have a working knowledge of this software
and the location of NEMIS data in the system in order to produce the
required reports.

### Configuration Control Management

NEMIS is a grouping of many different pieces of hardware platforms and
software programs. All releases of the system will be thoroughly tested
to operate in the FEMA environment on systems specifically configured to
support NEMIS. It is imperative that these hardware and software
configurations be documented and maintained so there is a standard
environment for operations and future releases of the system. This will
be accomplished by the O&M group using configuration control software, a
software change request system, and through the development of the
standard NEMIS environment.

### Local Area Network/Wide Area Network (LAN/WAN)

The key function of NEMIS is to provide useful data to the end-user.
This must be accomplished without regard to the physical location of the
user or the data. NEMIS will accomplish this using the FEMA LAN/WAN
infrastructure. Local data will be kept as current as possible using
Oracle's data base replication tools over the FEMA Switched Network
(FSN) and users will access the data over their LAN. When a user
requests data that does not originate locally or is not replicated to
the user's LAN, the user will access the data over the FSN as a remote
user. All of these data access methods must be monitored and maintained.
Routers on the FSN must be "tuned" for providing optimal transmission
routes and efficient use of the WAN. The LANs must be properly segmented
and maintained to provide adequate addressing, bandwidth, and error free
transmission of NEMIS data.

### Client Workstation Support

The NEMIS client workstation provides the user with all access and
functionality to the NEMIS data. It must support Oracle, ViewStar,
TCP/IP, all of the NEMIS client applications, the FEMA standard software
suite, and the end user's applications. O&M has the responsibility to
ensure that all of these applications work with each other. They must
also troubleshoot client side hardware and software problems.

### NEMIS Security

While NEMIS does not contain classified information, the system is
required to protect its information in accordance with the Computer
Security Act of 1987 and the Privacy Act. This means that access to the
NEMIS system must be controlled and that the data inside the system may
not be freely distributed. The development team has built logging and
auditing functions into the system in addition to those provided by the
COTS software.

The built-in security functions of NEMIS are being provided through a
COTS product called PadLock. PadLock is an application security system
that allows or disallows access to PowerBuilder applications, screens,
or fields on the screens via role based access controls (RBAC) assigned
to the NEMIS username.

In addition to the PadLock RBAC security, some NEMIS functions will also
use Oracle RBAC. Oracle's RBAC will be implemented against the Oracle
username and provide secure RBAC to the NEMIS data using roles,
privileges, and triggers. O&M has the responsibility of monitoring and
maintaining the logs generated by PadLock and Oracle to maintain system
integrity.

# General Concept of Operations

In its current state, FEMA maintains hundreds of server-centric LAN
environments that service the day-to-day e-mail, file, and print service
requirements of its user base. The local administrators are not forced
to follow networking standards and naming conventions developed by IT
policy makers. Simple conventions such as user names vary from site to
site. The results are "islands" of functional systems that operate
within themselves, but do not interact well as an enterprise system. If
NEMIS were implemented with the same lack of structure and standards, it
would be very likely that one NEMIS site would not be able to
communicate with another.

However, on the telecommunications side of FEMA, strict standards,
policies, and procedures have been adopted and followed. Management of
the overall system is approached from a centralized concept that allows
the administrators to keep control within a well-defined group. System
changes are effected centrally and cascaded out to remote sites. Local
administrators have limited access, but are capable of aiding the
central administrators and troubleshooting local system problems. This
allows FEMA to make best use of its manpower assets by maintaining a
core of high level support at the central site and minimizing the
requirement for highly skilled localized talent. This is the best
approach for an agency wide system like NEMIS.

The only way to maintain consistent data integrity and best utilize
FEMA's small knowledge pool across an agency wide system of this
magnitude is to implement a centralized enterprise management operation
for NEMIS. The following subsections identify the approach for the O&M
centralized concept (section 3.1) and a detailed description of the
centralized (section 3.3) and localized (section 3.2) functions
accomplished by this approach.

## Approach

To best support the implementation of NEMIS, FEMA O&M must implement a
tiered support architecture. The tiered structure must provide a
localized help desk, administrative support and access to a high-level
centralized technical support capability. This plan identifies three
primary support tiers: Tier One for general support; Tier Two for
administrative support; and Tier Three for high level technical support.
Tiers One and Two provide localized support and Tier Three provides
centralized support.

Tier One support provides general assistance to NEMIS users on operating
procedures and COTS software assistance. Tier One support reports
directly to the Tier Two administrators and administrative access to mid
or high-level system controls are very limited. It is very important to
understand that the Tier One support teams are the eyes and ears of the
O&M directorate as they interface daily with the customer. Tier One
personnel must be competent, customer oriented, and capable of
supporting the user environment. Tier Two is comprised of local
administration teams that provide day-to-day support to the end user and
act as a buffer between the user community and the Tier Three support
team. Tier Three support is the centralized team of Administrators
capable of managing any component of the NEMIS system.

Tier Three is further divided into three separate levels. As shown in
Figure 3-1 Tier Three Enterprise Support Levels, Tier Three is comprised
of a National Helpdesk (Level 1), an O&M administrative team (Level 2),
and an engineering/development support faction (Level 3).

![](media/image3.wmf)

[]{#__RefHeading___Toc419855762 .anchor}Figure 3‑1 Tier Three Enterprise
Support Levels

All support requests that can not be handled by Tiers One and Two will
be passed up to Tier Three, Level 1. Tier Three, Level 1 will pass the
support requests to the proper Tier Three, Level 2 administrative
support team. If for some reason, the support request can not be handled
at Tier Three, level 2, it will be coordinated and passed to the
engineering/development faction at Tier Three, Level 3. Figure 3-2
Overview of Support Request Flow Between Tiers provides a graphical
representation of this approach.

![](media/image4.wmf)

[]{#__RefHeading___Toc419855763 .anchor}Figure 3‑2 Overview of Support
Request Flow Between Tiers

### Tier One Support

Tier One support provides primary Helpdesk support at all NEMIS sites.
This team works directly for the Tier Two support team and performs all
of the basic duties required to keep a NEMIS site functional. The
primary skill set requirements for a Tier One support team member is a
functional knowledge of PCs, FEMA standard software, and the client side
of NEMIS. Tier One support mans the Helpdesk phones, enters Helpdesk
trouble tickets, and troubleshoots basic NEMIS problems over the phone.
Figure 3-3 Localized Support - Tier One Support Structure, shows the
placement of Tier One support within the localized support team. In the
figure, the unshaded area represents Tier One.

![](media/image5.emf)

[]{#__RefHeading___Toc419855764 .anchor}Figure 3‑3 Localized Support -
Tier One Support Structure

It is important to note that the Tier One support is O&M's primary
interface with the customer. While it is important to provide a stable
system through Tiers Two and Three, it is just as important to maintain
a high degree of user satisfaction through Tier One. Personnel
supporting Tier One must be competent, customer oriented, and
knowledgeable of their specific duties. Individual team members
providing Tier One support should be selected very carefully.

#### Overview of NEMIS Functions Supported

The following bulleted list identifies the overall responsibility of the
Tier One support team. Some of the specific functions associated with
these duties are explained later in this section. These duties are
provided for staff planning purposes and are as accurate as can be
estimated for a system that has not yet been implemented. As NEMIS is
implemented and O&M functions mature, the bullets below will be expanded
into separate sections that provide detailed descriptions of the
functions or reference other supporting NEMIS O&M documentation (i.e.,
administration manuals and SOPs).

-   Application installation, troubleshooting and support (Section
    3.2.5)

-   PC installation, troubleshooting and support (Section 3.2.5)

-   Printer installation, troubleshooting and support (Section 3.2.5)

-   Remedy Helpdesk phone support (Section 3.2.5)

-   Work with Tier Two to implement site hardware and software changes
    (Section 3.2.5)

#### Skill Set Required

In order to support the general functions described above, FEMA O&M will
be required to staff the Tier One support team with individuals that
have an acceptable level of skill and knowledge of the NEMIS
architecture. The following list identifies the general skill set the
team must posses. Section 4 of this document provides a skill set matrix
for each skill set identified below.

-   Helpdesk Technician

-   FEMA Application Specialist

-   PC Technician

#### Locations

Tier One support is required at all NEMIS sites.

### Tier Two Support

Tier Two support provides localized, mid-level O&M support at primary
NEMIS sites. This team coordinates localized operations with the
centralized administration team and provides for the needs of the local
customer. While the training and experience requirements for Tier Two
support is not as stringent as Tier Three, these individuals do require
a high level of practical NEMIS system experience and training. Tier Two
provides more Helpdesk-oriented support and interacts directly with the
NEMIS end users. Figure 3-4 Localized Support - Tier Two Support
Structure, shows the placement of Tier Two support within the localized
support team. The unshaded area represents the Tier Two support team.
Notice that the local site manager is included in the Tier Two
structure. For manning purposes, this position can be satisfied by one
of the other Tier Two positions (e.g., server or Oracle support).

![](media/image6.emf)

[]{#__RefHeading___Toc419855765 .anchor}Figure 3‑4 Localized Support -
Tier Two Support Structure

#### Overview of NEMIS Functions Supported

The following bulleted list identifies the overall responsibilities of
the Tier Two support team. Some of the specific functions associated
with these duties are explained later in this section. These duties are
provided for staff planning purposes and are as accurate as can be
estimated for a system that has not yet been implemented. As NEMIS is
implemented and O&M functions mature, the bullets below will be expanded
into separate sections that provide detailed descriptions of the
functions or reference other supporting NEMIS O&M documentation (i.e.,
administration manuals and SOPs).

-   NEMIS Server backups to include Oracle, NT, and ViewStar (Section
    3.2.1)

-   General Oracle report generation (Section 3.2.1)

-   NEMIS Server monitoring using BMC Patrol and other COTS tools
    (Section 3.2.1)

-   ViewStar queue monitoring (Section 3.2.2)

-   Server/LAN support (Section 3.2.3)

-   WAN support (Section 3.2.4)

-   Remedy Helpdesk support (Section 3.2.5)

More specific duties and functions will be identified as NEMIS becomes
operational.

#### Skill Set Required

In order to support the general functions described above, FEMA O&M will
be required to staff the Tier Two support team with individuals that
have an acceptable level of skill and knowledge of the NEMIS
architecture. The following list identifies the general skill set the
team must posses. Section 4 of this document provides a skill set matrix
for each skill set identified below.

-   Oracle Technician

-   ViewStar Technician

-   MS NT/Novell NetWare Administrator

-   LAN Administrator

-   WAN Coordinator

-   Help Desk Manager

#### Locations

Tier Two support is required at all NEMIS server sites that have a
medium to large user community. Therefore, Tier Two support will be
required at the following locations.

-   HQ FEMA

-   MWEAC

-   NETC

-   All Regions

-   Denton, Texas

-   Hyattsville, Maryland

-   All other medium and large FEMA supported sites

The list attempts to maximize FEMA's Tier Two personnel across the NEMIS
environment. However, support can be shuffled should the need for a Tier
Two technician arise at a small location.

### Tier Three, Level 1 Support

The NEMIS O&M Tier Three, Level 1 National Helpdesk will provide all
Tier One and Two teams with a centralized point of contact into the Tier
Three support structure. Tier three, Level 1 will use Remedy Helpdesk
system to enter trouble ticket calls into the system. Like the Tier One
Helpdesk, the Tier Three, Level 1 support will include initial trouble
ticket resolution activities. If the trouble ticket can not be solved in
a satisfactory period of time, the ticket will be forwarded to the Tier
Three, Level 2 support team. Figure 3-5 Centralized Support - Tier
Three, Level 1 Support Structure, shows the placement of Tier Two
support within the localized support team. The unshaded area represents
the Tier Three, Level 1 support team.

![](media/image7.emf)

[]{#__RefHeading___Toc419855766 .anchor}Figure 3‑5 Centralized Support -
Tier Three, Level 1 Support Structure

#### Overview of NEMIS Functions Supported

The following bulleted list identifies the overall responsibility of the
Tier One support team. Some of the specific functions associated with
these duties are explained later in this section. These duties are
provided for staff planning purposes and are as accurate as can be
estimated for a system that has not yet been implemented. As NEMIS is
implemented and O&M functions mature, the bullets below will be expanded
into separate sections that provide detailed descriptions of the
functions or reference other supporting NEMIS O&M documentation (i.e.,
administration manuals and SOPs).

-   Manage and maintain the National FEMA (Remedy) Helpdesk system
    (Section 3.2.5)

-   Develop and generate enterprise level helpdesk reports (Section
    3.2.5)

-   Act as a liaison between local and centralized support functions
    (Section 3.2.5)

-   Assist in the development of Enterprise wide SLAs (Section 3.2.5)

-   Provide general user assistance for any/all FEMA user issues
    (Section 3.2.5)

-   Provide referral to local helpdesk support team (Section 3.2.5)

-   Application troubleshooting and telephonic support (Section 3.2.5)

-   PC troubleshooting and telephonic support (Section 3.2.5)

-   National Helpdesk phone support (Section 3.2.5)

-   Work with Tier Two to coordinate site hardware and software changes
    (Section 3.2.5)

-   Work with Tier Three, Level 2 to coordinate national level trouble
    ticket support.

#### Skill Sets Required

In order to support the general functions described above, FEMA O&M will
be required to staff the Tier Three, Level 1 support team with
individuals that have an acceptable level of skill and knowledge of the
NEMIS architecture. The following list identifies the general skill set
the team must posses. Section 4 of this document provides a skill set
matrix for each skill set identified below.

-   Enterprise Helpdesk Manager

-   Enterprise Helpdesk Coordinator

-   Helpdesk Technician

-   FEMA Application Specialist

-   PC Technician

#### Tier Three, Level 1 Locations

Tier three, Level 1 support will be provided out of FEMA's Mount Weather
facility. This centralized support approach will allow FEMA O&M to best
utilize its high end NEMIS personnel. Furthermore, co-locating and
consolidating the Tier Three, Level 1 and 2 support for all of the
primary NEMIS functions will promote a more cohesive support of the FEMA
enterprise.

### Tier Three, Level 2 Support

The NEMIS O&M Tier Three, the most experienced and highly trained
individuals in the IT-OP Division must staff Level 2 support team. The
team will support the overall day-to-day functions of the FEMA
enterprise, identify enterprise level standard operating procedures,
oversee the implementation of NEMIS servers, and provide high level help
desk support to tiers one and two. These individuals will be responsible
for the most complex O&M duties in the NEMIS system. This group of
experienced and highly trained individuals will form the Enterprise
Systems Management (ESM) team.

![](media/image8.emf)

[]{#__RefHeading___Toc419855767 .anchor}Figure 3‑6 Centralized Support -
Tier Three, Level 2 Support Structure

The ESM is the core of the of centralized NEMIS support. In addition to
the day-to-day Tier Three support duties, the ESM team members are
responsible for working with FEMA senior O&M managers and the IT,
Management Division, Policy and Requirements Branch to identify SOPs for
NEMIS and its support environment. The ESM team will also work directly
with the IT Engineering Division to identify short and long term
requirements for the NEMIS environment. The reporting structure for the
ESM team should be as high as possible in the O&M management chain to
avoid unnecessary delays or political obstacles that may slow the team's
response process.

#### Overview of NEMIS Functions Supported

The following bulleted lists identify the general responsibilities of
the ESM team and the Tier Three support team. The first set of bullets
represents the duties that are performed solely by the ESM team members.
The second set of bullets is the responsibilities of all Tier Three Team
members.

Some of the specific functions associated with these duties are
explained later in this section. These duties are provided for staff
planning purposes and are as accurate as can be estimated for a system
that has not yet been implemented. As NEMIS is implemented and O&M
functions mature, the bullets below will be expanded into separate
sections that provide detailed descriptions of the functions or
reference other supporting NEMIS O&M documentation (i.e., administration
manuals and SOPs).

This list represents the general duties of the ESM Team members.

-   Management of the Tier Three, Levels 1 and 2 support teams (Section
    3.3.1)

-   Work with PMG to plan, test and coordinate all NEMIS system changes
    (Section 3.3.1)

-   Develop SOPs for all NEMIS systems and subsystems (Section 3.3.1)

-   Work with IT policy makers to define, implement, and enforce
    standards (Section 3.3.1)

-   Work with PMG to identify and review system requirements (Section
    3.3.1)

-   Train the Tier Three, Level 1 and 2 support team on high level
    system functions (Section 3.3.1)

```{=html}
<!-- -->
```
-   Plan for the NEMIS O&M system life cycle (Section 3.3.1)

In addition to the ESM functions identified above, the Tier Three, Level
2 support team will also perform the following.

-   High level Oracle database support, recovery, planning, and
    implementation (Section 3.3.3)

-   High level ViewStar support, recovery, and planning, and
    implementation (Section 3.3.4)

-   High level WAN maintenance administration, recovery, and planning
    (Section 3.3.5)

-   High level server/LAN maintenance support, recovery, and planning
    (Section 3.3.6)

-   Support and monitor NEMIS system configurations (Section 3.3.7)

-   Enterprise security monitoring and administration (Section 3.3.8)

-   High Level WWW Maintenance Support and Planning (Section 3.3.9)

-   High/Enterprise level Helpdesk issues (Section 3.3.10)

More specific duties and functions will be identified as NEMIS becomes
operational.

#### Skill Sets Required

In order to support the general functions described above, FEMA O&M will
be required to staff the Tier Three support team with individuals that
have an acceptable level of skill and knowledge of the NEMIS
architecture. The following list identifies the general skill set the
team must posses. Section 4 of this document provides a skill set matrix
for each skill set identified below.

-   Enterprise Oracle DBA

-   Oracle DBA

-   Enterprise ViewStar Administrator

-   ViewStar Administrator

-   Enterprise WWW Webmaster

-   Senior MS NT/Novell NetWare Enterprise Administrator

-   MS NT/Novell NetWare Enterprise Administrator

-   Enterprise LAN/WAN Administrator

-   WAN Administrator

-   LAN Support

-   NEMIS System Security Support Team

-   NEMIS Configuration Control Support Team

The skill sets required to support NEMIS may be expanded as NEMIS
becomes operational and additional O&M functions are identified.

#### Tier Three, Level Two Locations

The centralized support approach will allow FEMA O&M to best utilize its
high end NEMIS personnel. Furthermore, co-locating and consolidating the
Tier Three, Level 1 and 2 support for all primary NEMIS functions will
promote more cohesive support of the FEMA enterprise. Tier Three, Level
2 support will be provided out of FEMA's Mount Weather facility.

### Tier Three, Level 3 Support

Tier Three, Level 3 of the NEMIS support structure is the final, in
house support structure within FEMA. This level of support is manned by
the engineers and developers that architect the systems and software
that make up NEMIS. Trouble tickets that are escalated to this level of
support have passed through the entire support system and may require
modification to a system or subsystem within NEMIS. Figure 3-7
Centralized Engineering/Development - Tier Three, Level 3 Structure,
shows the placement of Tier Three, Level 3 in the NEMIS support
structure. Note that the unshaded area represents the Tier Three, Level
3 team.

![](media/image9.wmf)

[]{#__RefHeading___Toc419855768 .anchor}Figure 3‑7 Centralized
Engineering/Development - Tier Three, Level 3 Structure

The day-to-day duties of Tier Three, Level 3 are primarily engineering
and development. However, they will be requested to assist the O&M
support structure when required. As this document is specific to the O&M
functions of NEMIS, it will only discuss the Tier Three, Level 3 support
of O&M functions. It is assumed that the ENG and PMG have documentation
that supports and guides their engineering and development functions.

#### Overview of NEMIS Functions Supported

The following bulleted lists identify the general responsibilities of
the Tier Three, Level 3 teams. Again, these duties are specific to the
engineers and developers in the support of the O&M teams. The day-to-day
engineering and development tasks performed by these groups should be
identified in separate documentation.

The following duties are provided for planning purposes and are as
accurate as can be estimated for a system that has not yet been
implemented. As NEMIS is implemented and O&M functions mature, the
bullets below will be expanded by the supporting groups (ENG, MA, and
PMG) to provide detailed descriptions of the functions or reference
other supporting NEMIS documentation (i.e., engineering diagrams or
software design documentation).

-   Work with O&M to plan, test and coordinate all NEMIS system changes;

```{=html}
<!-- -->
```
-   NEMIS system development life cycle;

-   Gather and Analyze NEMIS system requirements;

-   Develop NEMIS system components, enhancements, and patches;

-   Test NEMIS system components, enhancements, and patches;

```{=html}
<!-- -->
```
-   Final in house support for Oracle database trouble resolution;

```{=html}
<!-- -->
```
-   Development of enterprise wide Oracle ad hoc reports and canned
    system reports;

-   Final in house support for the NEMIS ViewStar system issues;

```{=html}
<!-- -->
```
-   Modification of ViewStar workflows and maps;

-   Final in house support for WAN trouble ticket issues;

-   Final in house support for LAN and server trouble ticket issues;

-   System configuration management and control;

-   NEMIS version and release control;

-   Define NEMIS system hardware baselines;

-   Enterprise security determination and standards;

-   Final in house support for WWW trouble ticket issues;

-   Web page design and development; and

-   Provide support to all O&M support areas as required.

#### Skill Sets Required

In order to support the general functions described above, FEMA ENG, MA,
and PMG will be required to staff their Tier Three, Level 3 support
teams with individuals that have an acceptable level of skill and
knowledge of the NEMIS architecture. The following list identifies the
general skill set that is recommended for each area. However, each
faction has other primary engineering and development tasking that may
overlap with the skill sets identified. Therefore, each faction should
review their staffing requirements across their entire engineering and
development efforts and provide a complete skill matrix that covers all
of their respective efforts. This matrix should also identify skill set
requirements that over lap with those identified below. As these
functions fall outside of the O&M support requirement, they are
identified but expanded upon in Section 4 of this document. Furthermore,
the training requirements for these positions are not covered by this
document.

-   Oracle Database Design Engineer

-   Oracle DBA

-   Oracle Report Designer

-   ViewStar System Design Engineer

-   Web Designer

-   Senior MS NT/Novell NetWare Enterprise Administrator

-   Senior WAN Administrator

-   NEMIS System Security Manager

-   NEMIS Configuration Control Manager

#### Tier Three, Level 3 Locations

While it would be advantageous to co-locate the Tier Three, Level 3
support functions with the ESM, it may not be feasible given that the
primary duties of Tier Three, Level 3 are engineering and development.
Furthermore, as NEMIS and the O&M support structure matures, there will
be fewer requests for assistance to the Tier Three, Level 3 support
factions. Therefore, each faction should establish a primary point of
contact for each functional area in Tier Three. This point of contact
may be telephonic or Helpdesk driven and could be located at Mount
Weather or FEMA Headquarters.

### Mobile/Remote Support Teams

NEMIS has been developed to support all FEMA disaster support personnel
at all disaster support locations. This includes users at DFOs or remote
field offices. In the case of emergency support, NEMIS sites must be
implemented as quickly as possible to support the DFO mission.
Therefore, O&M must be capable of responding to the support requirements
of these remote NEMIS locations. However, this capability should not be
provided at the expense of fixed support sites. FEMA should utilize a
cadre of NEMIS trained IT disaster support personnel, Mobile Emergency
Response System (MERS) technicians and Disaster Temporary Employees
(DTEs) to accomplish this mission. MERS ADP staff could be trained as
the Tier Two support and use the DTEs as the Tier One support.

#### Overview of NEMIS Functions Supported

Initial NEMIS sites will have the same functional requirements as Tier
Two and Three support sites.

#### Skill Set Required

MERS teams will support Tier Two functions and therefore, will require
skills equivalent to Tier Two support. The DTEs will support Tier One
functions and therefore must possess Tier One skills.

#### Locations

The mobile team will support the emergency disaster response
requirements of the DFOs and remote field offices. They will work
directly with the Tier Three support team and the IT Disaster Support
staff to implement and provide short-term support for NEMIS disaster
support servers.

## Localized Administrative Functions (Tier 1 & 2)

A centralized approach to support does not eliminate the need to provide
support at a local site. It does minimize the requirement for high-end
support. However, because local support is the O&M interface to the
customer, a high level of expertise coupled with inter-personal skills
are required. This is the justification for a two-tier approach at the
local site.

Local support must be competent, capable of working with end users in a
friendly and timely manner, and provide an interface to the centralized
team. The following subsections identify some of the localized team
duties. As NEMIS is implemented and O&M functions mature, the
subsections below will be expanded into separate sections that provide
detailed descriptions of the functions or reference other supporting
NEMIS O&M documentation (i.e., administration manuals and SOPs).

### Localized Oracle RDBMS Database Administrative Functions (Tier 2)

The centralized Oracle management team will perform most Oracle
administrative functions. Localized support will be limited to the
following duties:

-   Knowledge of the local Remedy helpdesk system;

-   Understanding of the local helpdesk trouble ticket and escalation
    policies;

```{=html}
<!-- -->
```
-   Interface with and assist the centralized Oracle management team;

-   Use BMC PatrolWatch to monitor the Oracle instance;

-   Assist in the backup/recovery of the NEMIS system with LAN/Server
    support;

-   Execute local NEMIS (Oracle) system reports;

```{=html}
<!-- -->
```
-   Troubleshoot Oracle workstation problems;

-   Help develop "Service Level Agreements" (SLAs) with the user
    community and Tier Three;

```{=html}
<!-- -->
```
-   Maintain local NEMIS reference data; and

-   Provide mid level Helpdesk support for local Oracle users.

### Localized ViewStar Administrative Functions (Tier 2)

The local ViewStar Administrator may require more hands on support of
the ViewStar system. This is due to ViewStar's lack of an enterprise
management strategy in a very large ViewStar installation. However,
NEMIS uses ViewStar as an embedded application. NEMIS users do not
directly interface with ViewStar client software. Most duties associated
with local administration will be monitoring the ViewStar environment
for signs of trouble and working with the centralized management team to
resolve problems. Localized ViewStar support consists of:

-   Knowledge of the local Remedy helpdesk system;

-   Understanding of the local helpdesk trouble ticket and escalation
    policies;

```{=html}
<!-- -->
```
-   Interface with and assist the ViewStar centralized management team;

```{=html}
<!-- -->
```
-   Assist in the backup/recovery of the NEMIS system with LAN/Server
    support;

```{=html}
<!-- -->
```
-   Use BMC PatrolWatch to monitor the ViewStar server processes

-   Maintain ViewStar space allotments;

-   Manage online ViewStar documentation;

-   Execute local ViewStar system reports;

```{=html}
<!-- -->
```
-   Help develop "Service Level Agreements" (SLAs) with the user
    community and Tier Three;

```{=html}
<!-- -->
```
-   Troubleshoot ViewStar workstation problems; and

-   Provide mid level Helpdesk support for ViewStar users.

### Localized Server/LAN Administration (Tier 2)

Localized Server/LAN administrative support is required to keep the
NEMIS environment operational. It is the responsibility of the local
support to ensure the smooth operation of the NEMIS servers,
workstations, and networks.

The local Server/LAN support team is the eyes and ears of the entire
NEMIS IT-OP support structure. The local team is not only responsible
for the day-to-day operation of the environment, but it must also plan
and coordinate all system-related issues with the local customer and the
centralized support team. Localized Server/LAN support consists of:

-   Knowledge of the local Remedy helpdesk system;

-   Understanding of the local helpdesk trouble ticket and escalation
    policies;

```{=html}
<!-- -->
```
-   Interface with and assist the local customer;

-   Interface with and assist the centralized server/LAN administrative
    team;

```{=html}
<!-- -->
```
-   Help develop "Service Level Agreements" (SLAs) with the user
    community and Tier Three;

```{=html}
<!-- -->
```
-   Use Tivoli tool to create, modify, and delete NEMIS user accounts;

-   Use BMC PatrolWatch and other system provided utilities to monitor
    the NEMIS NT server environment;

-   Plan, coordinate, and execute NEMIS server backup and recovery;

-   Plan, coordinate and execute the installation of NEMIS client PCs;

-   Plan, coordinate and execute the installation of NEMIS support tools
    in the environment;

-   Perform local hardware preventative maintenance;

-   Perform local hardware repairs and updates;

-   Assist in the automated distribution of updates to the NEMIS client;

-   Use local LAN support tools to track and monitor the local LAN
    environment;

-   Configure, implement, and monitor NEMIS client Novell NetWare
    network printer support;

-   Troubleshooting local cable plant problems;

-   Day-to-day support of the local FEMA office automation network
    users, servers, and workstations (NetWare);

-   Monitor and maintain the local site computer environment;

```{=html}
<!-- -->
```
-   Help develop "Service Level Agreements" (SLAs) with the user
    community and Tier Three;

```{=html}
<!-- -->
```
-   Enforce enterprise LAN, workstation, and server standards and
    policies.

### Localized WAN Support Functions (Tier 2)

Localized WAN support is required to provide physical support and
assistance to the centralized WAN management team. As most of the
monitoring and maintenance of the WAN/router environment will be
performed from a central location, the duties performed under this title
could be rolled into the localized Server/LAN support team functions.
Localized WAN support consists of:

-   Knowledge of the local Remedy helpdesk system;

-   Understanding of the local helpdesk trouble ticket and escalation
    policies;

```{=html}
<!-- -->
```
-   Physical monitoring of the local router;

-   Assist centralized WAN administrator troubleshoot routers and;

```{=html}
<!-- -->
```
-   Help develop "Service Level Agreements" (SLAs) with the user
    community and Tier Three;

```{=html}
<!-- -->
```
-   Enforce WAN standards and policies.

### Localized Help Desk Support (Tier 1 & 2)

As previously stated, the local helpdesk is the front line of the IT-OP
support structure. Individuals on this team are the direct interface to
the NEMIS customer. Therefore it is essential that these individuals be
well trained and qualified in their primary support skills. Localized
helpdesk support consists of:

-   Knowledge of the local Remedy helpdesk system;

-   Understanding of the local helpdesk trouble ticket and escalation
    policies;

```{=html}
<!-- -->
```
-   Manage site Helpdesk support;

```{=html}
<!-- -->
```
-   Application installation, troubleshooting and support;

-   Execute PC installation, troubleshooting and support;

-   Maintain local printer installations, troubleshooting and support;

-   Provide Helpdesk phone support to the user community;

```{=html}
<!-- -->
```
-   Work with Tier Two to implement client hardware and software
    changes;

-   Help develop "Service Level Agreements" (SLAs) with the user
    community;

-   Track site Helpdesk functions, technician utilization, and trouble
    tickets; and

```{=html}
<!-- -->
```
-   Generate site Helpdesk reports.

## Centralized Administrative Functions (Tier 3, Level 1 & 2)

Centralizing FEMA's O&M operation and support will allow FEMA to best
utilize its personnel and administrative software. It will also provide
a central site for developing standards for operation. Both of these
benefits bolster FEMA's push to implement an enterprise based O&M
support system.

The following subsections identify centralized approaches and team
duties. The approach descriptions identify how the overall O&M function
fit into a single, cohesive environment. The duties identified provide a
more detailed description of the overviews provided in section 3.1.1.1,
Overview of (Tier Three) NEMIS Functions Supported. As NEMIS is
implemented and O&M functions mature, the subsections below will be
expanded into separate sections that provide detailed descriptions of
the functions or reference other supporting NEMIS O&M documentation
(i.e., administration manuals and SOPs).

### Centralized Enterprise Management Approach

Simply centralizing O&M functions does not provide an enterprise
solution to system operation, maintenance, and/or administration. The
Tier Three, Level 2 support environment must be closely associated and
reliant upon each other to provide a complete support solution. The use
of monitoring and administration tools and access to these tools must be
planned for the entire enterprise environment. For this reason, IT-OP
has implemented a NEMIS Operation Center. All Tier Three, Level 2
support personnel will co-located in the same general area at Mt.
Weather and they will use a common set of monitoring tools to maintain
the NEMIS environment.

IT-PM has provide a complete set of tools to manage the NEMIS
environment. The local and central administrative teams to monitor the
NT, Oracle, Exchange, ViewStar, and NEMIS (Human Services) systems will
use BMC Patrol. Tivoli User Administration services will be used by
local and centralized teams to manage the NEMIS user and security
environment. Microsoft's System Management Server will be used to
automate the distribution of NEMIS software and maintain a NEMIS client
inventory.

#### BMC Patrol

BMC Patrol will be used to monitor the entire NEMIS environment. Out of
the box Patrol is capable of monitoring 100+ NT server function, 75+
Exchange functions, and 150+ Oracle RDBMS functions. IT-PM has
customized the BMC Patrol monitoring capability to perform ViewStar and
some NEMIS specific monitoring functions (i.e., Human Services). This
overall system approach for NEMIS monitoring will allow the IT-OP
centralized management team to quickly and accurately react to problems.

For example, if the BMC Patrol monitors on the Oracle, Server/LAN and
ViewStar administrator's systems all indicate that those systems have
failed, it is likely that the WAN connection to the site is down. This
could quickly be confirmed through the WAN administration team. Without
a coordinated troubleshooting effort, the IT-OP support team might have
wasted time checking out localized system problems before looking to a
larger WAN type problem. Furthermore, without a centralized monitoring
tool, they might have waited for a user initiated complaint.

The general approach for implementing BMC Patrol in the NEMIS
environment is provided in Appendix A.

#### Tivoli User Administration

NEMIS is a client server application that is based on several COTS
systems. MS NT is the base operating system of the server environment.
Oracle RDBMS is the data repository for NEMIS information and provides
some NEMIS system security. PadLock is used to provide application level
role based access controls. The NEMIS security subsystem uses all of
these environments to extend NEMIS management control security. All
three of these NEMIS components require some level of user management.

Without a common user management tool IT-OP would have to manage each of
these components as a separate system/subsystem. Therefore, Tivoli was
procured and customized by IT-PM to provide a single point of user
management for these systems/subsystems.

Tivoli is a client server based system. The client is used to manage
users in a centrally located database. Localized administrators will use
a customized Tivoli desktop to create, modify, and/or delete users in
their specific environment (e.g., Region VI administrator manages Region
VI users). The centralized administrators will manage the Tivoli
database and all of the localized administrative user accounts.

The server side of the Tivoli system will store the user information
provided by the local administrators and distribute it to the target
systems. The distribution process actually performs the user creation,
modification, and/or deletion.

The general approach for implementing Tivoli User Administration in the
NEMIS environment is provided in Appendix C.

#### MS System Management Server (SMS)

As previously stated, NEMIS is a client server system. Each user of the
NEMIS system must use a client side front-end to operate in the
environment. NEMIS is also a distributed system. Clients can and
probably will be located anywhere where there is a FEMA location or
access requirement (e.g., state and/or local government). All of these
clients must be upgraded as new releases and patch fixes are made
available. IT-PM purchased SMS to automate this process.

The general approach for implementing MS System Management Server in the
NEMIS environment is provided in Appendix C.

### Overall Centralized NEMIS Management Functions (Tier 3, Level 2 and Senior Management)

While NEMIS has a functional support requirement in the FEMA O&M
environment, it also directly impacts several FEMA wide requirements and
senior management level functions. The ESM and Tier Three support teams
will be primarily responsible for these management level functions. The
following bullets identify some of these overall functions and
management areas that must be addressed from a centralized approach.

-   Maintain the Concept of Operations and all other support
    documentation;

-   NEMIS disaster server planning and setup;

```{=html}
<!-- -->
```
-   Pre-load the NT and Novell servers

-   Pre-load the NEMIS Oracle databases

-   Pre-load the ViewStar file systems and databases

-   Pre-load the NEMIS COTS applications and databases

-   Pre-load the NEMIS support systems

-   Setup the NEMIS Oracle routing databases

-   Coordinate implementation with IT Engineering and Mobile Support
    Teams

```{=html}
<!-- -->
```
-   Coordination with FEMA disaster support functions;

```{=html}
<!-- -->
```
-   Initialize replications after implementation

-   Coordinate ViewStar queues with NEMIS databases

-   Download/Snapshot database updates

```{=html}
<!-- -->
```
-   Identify O&M system support requirements;

```{=html}
<!-- -->
```
-   Collect and track System Change Requests (SCRs)

-   Work directly with PMG staff to identify short and long term
    requirements

-   Develop O&M scripts and procedures for day-to-day maintenance of the
    enterprise

```{=html}
<!-- -->
```
-   Plan NEMIS O&M life cycle;

```{=html}
<!-- -->
```
-   Monitor and track all aspects of the NEMIS environment

```{=html}
<!-- -->
```
-   Processor utilization

-   Disk utilization

-   Memory utilization

-   Oracle table space

-   ViewStar queues and cache

-   Develop detail monthly, quarterly, and yearly reports based on
    historical and trend data collection.

-   Use reports to plan and justify future NEMIS environment
    expenditures

```{=html}
<!-- -->
```
-   Lifecycle Hardware reporting and replacement/upgrade planning

-   System hardware expansion

```{=html}
<!-- -->
```
-   Develop detailed training plan for Tier Two and Three personnel;

```{=html}
<!-- -->
```
-   Structure a plan for long term organized training (commercial
    instruction, video tapes, books) supported by On the Job Training
    (OJT) by peers

-   Include training requirements in Tier Two and Three performance
    reviews

-   Identify quantity and cost requirements for recurring organized
    training

-   Include recurring training costs in long term budget requirements

```{=html}
<!-- -->
```
-   Evaluate FTE and DTE resources in the NEMIS environment;

```{=html}
<!-- -->
```
-   Review current staffing and assignments

-   Determine best utilization in new NEMIS environment

-   Request a formal, professional staffing review after NEMIS has been
    "full-up" operational for several months.

### Centralized Oracle Database Administrative Functions (Tier 3, Level 2)

Every NEMIS site will require access to NEMIS information via an Oracle
RDBMS. Centralized management of these resources is required to insure
data availability and integrity. Given the NEMIS requirement for data to
be distributed and available to any NEMIS user regardless of physical
location, NEMIS Oracle database management will be a very difficult
undertaking. To adequately monitor and maintain the NEMIS Oracle
environment the centralized Oracle support team must fully understand
the operational NEMIS Oracle environment.

While the enterprise management of the NEMIS Oracle environment is a
large task, most of the tools necessary to achieve centralized
management are provided through Oracle's Enterprise Management tool and
BMC Patrol monitors. Like most administrative tools, both the enterprise
management and BMC Patrol tools will require the development of special
NEMIS specific configurations to automate the management process.

The BMC Patrol tool will allow the centralized administration team to
simultaneously monitor all instances of Oracle in the NEMIS environment.
Problems or potential problems will be identified at the site level. The
centralized administrator can then "drill down" into the site and view
the problem at the instance and or Oracle component level.

The following bulleted list identifies some of the specific duties
performed by the Tier Three, Level 2 Oracle Administrators.

-   NEMIS Oracle technical architecture;

```{=html}
<!-- -->
```
-   Provide high level support to local Oracle support teams

-   Physical database configuration

-   NEMIS schema design

-   Replication architecture

-   Replication internals

-   Replication groups

-   Network configuration

```{=html}
<!-- -->
```
-   SQL\*Net listener configuration

-   Global instance names

-   Database links

```{=html}
<!-- -->
```
-   Installations and upgrades;

```{=html}
<!-- -->
```
-   Oracle software installation

-   Oracle database creation

-   NEMIS database creation

```{=html}
<!-- -->
```
-   Tablespaces

-   Roles

-   Schemas

-   Database Links

-   User accounts

```{=html}
<!-- -->
```
-   Import certified reference tables

```{=html}
<!-- -->
```
-   Default tablespace;

-   Temporary tablespace;

-   Default roles;

-   Passwords;

```{=html}
<!-- -->
```
-   Synonyms (public vs. private);

```{=html}
<!-- -->
```
-   Oracle software/database upgrades

-   Installation and configuration of replication support

-   NEMIS installation and upgrades

```{=html}
<!-- -->
```
-   Backup and recovery;

```{=html}
<!-- -->
```
-   Database backup procedures

```{=html}
<!-- -->
```
-   Cold database backup

-   Hot database backup

-   Offsite, online backups (between NPSCs)

-   Backup media handling

```{=html}
<!-- -->
```
-   Database recovery procedures

```{=html}
<!-- -->
```
-   Instance failure

-   Media failure

-   DBA recovery drills

```{=html}
<!-- -->
```
-   Failure procedures

```{=html}
<!-- -->
```
-   Database recovery at alternate site

-   Connection request redirection

-   Failover testing

```{=html}
<!-- -->
```
-   Space management and planning;

```{=html}
<!-- -->
```
-   Object growth characteristics

-   Schema reorganization

```{=html}
<!-- -->
```
-   Monitoring and tuning;

```{=html}
<!-- -->
```
-   Ongoing performance monitoring

-   Database tuning

-   Application tuning

-   Network tuning

```{=html}
<!-- -->
```
-   Replication management;

```{=html}
<!-- -->
```
-   Building the replicated environment

-   Quiescing the environment

-   Maintaining replication sites

-   Monitoring replication performance

-   Recovering from replication errors

```{=html}
<!-- -->
```
-   Security and user account maintenance;

```{=html}
<!-- -->
```
-   Account creation

-   Global userids

-   Initial password

-   Default roles

-   Profile

```{=html}
<!-- -->
```
-   Migration planning;

```{=html}
<!-- -->
```
-   NEMIS database upgrades

-   Migration window

-   Migration test and verification

-   Fallback procedures

```{=html}
<!-- -->
```
-   Enterprise manager operations; and

```{=html}
<!-- -->
```
-   Coordinate with the ViewStar, WAN, LAN, Configuration Control, and
    Security Teams to enact system level changes and ensure smooth
    operations.

### Centralized ViewStar Administrative Functions (Tier Three, Level 2)

NEMIS workflow maps, process agents, the ViewStar queue and file system,
and the ViewStar Oracle tables all makeup the NEMIS ViewStar system.
Centralized ViewStar administration will be accomplished via the FEMA
WAN.

IT has customized the BMC Patrol monitor to watch key functions of the
ViewStar system. The ViewStar Enterprise Management Team will use the
Patrol monitor to quickly identify when a ViewStar process agent has
failed. At this point, the administrator will login to remote ViewStar
systems to maintain the NEMIS ViewStar environment or he/she will
contact the local support for hands on assistance.

The ViewStar Enterprise Management Team will perform the following
tasks:

-   General administration of the ViewStar system;

-   Maintain workflow maps;

-   Use BMC to monitor all process agents;

-   Monitor and repair ViewStar file systems and Oracle tables;

-   Monitor the WEB server and imaging system interfaces;

-   Monitor the MS exchange transport services;

-   Maintain client side business process interface (BPI) for Power
    Builder;

-   Repair damaged ViewStar tables and file systems;

-   Maintain and purge ViewStar tracking tables;

-   Execute enterprise ViewStar system reports;

-   Plan and monitor ViewStar backup processes;

-   Provide high level support to local ViewStar support teams; and

-   Coordinate with the Oracle, WAN, LAN, Configuration Control, and
    Security Teams to enact system level changes and ensure smooth
    operations.

### Centralized WAN Administrative Functions (Tier 3, Level 2)

NEMS requirements for remote login, dial-up access, and database
replication will impact the FEMA WAN. The Enterprise WAN Management Team
must continually monitor and manage the WAN environment to provide the
fastest most secure transport for the NEMIS environment. This
requirement will require the team to perform the following:

-   Execute WAN changes;

-   HP OpenView enterprise monitoring;

-   Router monitoring and maintenance;

-   TACACS management;

-   Modem pool management;

-   Remote control PC setup and management;

-   WAN bandwidth utilization/error monitoring;

-   WAN troubleshooting;

-   Assist in the development of WAN standards and policies;

-   Monitor and control Cisco protocol routing;

-   Maintain dynamic IP address name resolution via DNS;

-   IP address management and control; and

```{=html}
<!-- -->
```
-   Coordinate with the Oracle, ViewStar, LAN, Configuration Control,
    and Security Teams to enact system level changes and ensure smooth
    operations.

### Centralized Server/LAN Administrative Functions (Tier 3, Level 2)

The implementation of NEMIS includes the installation of MS NT servers.
Under their current version of NT server Microsoft implements an
enterprise level management strategy based on NT domains. The NEMIS
environment will be based on a single NT domain called NEMIS.

The Tier Three, Level 2 Server/LAN Administration Team, will centrally
manage the NEMIS Domain. Localized administrative accounts will be
established to perform system backups and hardware maintenance. These
localized administrators will not have access to the Domain
administrative privileges.

BMC Patrol will be installed on all NEMIS NT systems and configured to
monitor specific key components of the system. Both the localized and
centralized administrative teams will have access to the BMC NT Monitor
information.

The following list identifies some of the duties of the centralized
server/LAN administrative functions:

-   NEMIS server monitoring using BMC Patrol;

-   Plan and monitor NT server installations;

-   Oversee the implementation of NEMIS in the NT environment;

-   Maintain a consistent hardware and software baseline environment for
    the NT systems;

-   Plan and monitor NetWare server installations;

-   Plan, implement and maintain NetWare Directory Services;

-   Provide high end Helpdesk support for servers and end-users;

-   Assist in the development of LAN standards and policies; and

```{=html}
<!-- -->
```
-   Coordinate with the Oracle, ViewStar, WAN, Configuration Control,
    and Security Teams to enact system level changes and ensure smooth
    operations.

### Centralized Configuration Management Support Functions (Tier 3, Level 2)

The NEMIS environment includes specific versions of the NEMIS system and
COTS software and hardware. It is important that all of these components
operate at same level to ensure system and data integrity.

The NEMIS Configuration Manager will work closely with the Tier Three,
Level 3 NEMIS configuration manager to maintain a baseline for NEMIS
software environment. MS SMS software distribution tool will be used to
maintain the NEMIS client environment. Tier Three, Level 3 Configuration
Manager will provide any updates or patches to the Tier Three, Level 2
Configuration Support team. An automated distribution job will be
developed and SMS will automatically distribute the new software to the
NEMIS client PCs.

The Tier Three, Level 2 support will also maintain server baseline
documentation and control the release of server updates in the
environment.

The following list identifies some of the specific support functions the
team will provide:

-   Identify and document NEMIS hardware and software baseline
    configuration information;

-   Publish baseline configuration information;

-   Manage the NEMIS SMS environment;

-   Manage and execute NEMIS client automatic software distribution;

-   Maintain the SMS SQL Server environment;

-   Maintain the SMS PC hardware and software inventory system;

-   Generate SMS reports;

-   Monitor and maintain distribution of COTS software updates, patches
    and upgrades;

```{=html}
<!-- -->
```
-   Novell NetWare

-   MS NT Server

-   MS NT Workstation

-   MS Windows 95

-   Oracle

-   ViewStar

-   MS Office 97

-   WEB browsers

-   Anti-Virus software

```{=html}
<!-- -->
```
-   Coordinate with the Oracle, ViewStar, WAN, LAN, and Security Teams
    to enact system level changes and ensure smooth operations.

### Centralized Security Support Functions (Tier 3, Level 2)

The NEMIS system receives input for disaster victims that has been
identified as sensitive but not classified. This information must be
protected as described by the Privacy Act, Computer Security Act, and
the FEMA Information Resource Management Policies and Procedures
Directive (FIRMPD). Therefore, the NEMIS Security Manager will be
responsible for the following duties:

-   Assist the Enterprise Security Manager to develop and maintain
    system security policies and procedures;

-   Interface with physical and computer security representatives;

-   Monitor, review and control NEMIS security log files;

-   Enforce security policies and procedures; and

-   Work with the Oracle, ViewStar, WAN, LAN, and Configuration Control
    Teams to enact system level changes and ensure smooth operations.

### Centralized WWW Administrative Functions (Tier 3, Level 2)

The NEMIS WWW server is based on Oracle's WWW server. This server
provides special links to the NEMIS Oracle data base servers. The
subsystems designed on the NEMIS Intranet server will use these special
links to control data in the NEMIS system. Therefore, the functions
performed by the NEMIS WWW Administrator must also include interaction
with the Oracle data base team to facilitate smooth interaction. The
initial release of NEMIS will provide one WWW server. The duties listed
below are functions that must be performed by the NEMIS WWW
Administrator:

-   Administer the WWW server;

```{=html}
<!-- -->
```
-   Maintain NEMIS internal and external WWW links; and

-   Coordinate with FEMA intranet WWW Administrators.

### National Helpdesk Support (Tier 3, Level 1)

Tier Three Helpdesk support provides the highest level of internal NEMIS
assistance available to the NEMIS end user. While Tier Three will
respond to Helpdesk trouble tickets from Tiers One and Two, their
primary Helpdesk support will concentrate on tickets that require
centralized support for resolution. This includes application support
for the NEMIS users. Tier Three, Level 1 helpdesk support will be
trained in the use of all NEMIS subsystems.

The list below identifies the areas where Tier Three will support
Helpdesk functions:

-   Manage and maintain the Enterprise Remedy helpdesk system;

-   Handle trouble tickets that require centralized support for
    resolution;

-   Route requests for assistance to the proper local helpdesk location;

-   Trouble tickets that generate a requirement for a change of the
    NEMIS system; and

-   Request for assistance from a Tier Two Administrator.

# Skill Set Matrix

The following table provides a lost of the skills required to provide
FEMA O&M with competent and capable support for the NEMIS environment.
The table identifies formal education, training, and relative experience
as the prime requirements for the NEMIS support team. However, it is
important to note that these are only guidelines and that each
individual considered for the NEMIS support positions should posses
experience that is relevant to FEMA's NEMIS O&M requirements. All
applicants [must be]{.underline} interviewed and their specific
capabilities and competence analyzed before being considered for
positions.

+--------------+---------+---------+---------+--------+------+-------+
| TiER Three   | R       |         |         |        |      |       |
|              | elative |         |         |        |      |       |
| POSITION     | E       |         |         |        |      |       |
|              | ducatio |         |         |        |      |       |
|              | n/Train |         |         |        |      |       |
|              | ing/Exp |         |         |        |      |       |
|              | erience |         |         |        |      |       |
|              | (Centr  |         |         |        |      |       |
|              | alized) |         |         |        |      |       |
+==============+=========+=========+=========+========+======+=======+
|              | Formal  | R       | R       | Oracle | VIEW | NT/NW |
|              | Ed      | elative | elative | Exp    | STAR |       |
|              |         |         | Exp     |        |      | Exp   |
|              |         | T       |         |        | Exp  |       |
|              |         | raining |         |        |      |       |
+--------------+---------+---------+---------+--------+------+-------+
| Enterprise   | BS      | Oracle  | 10      | 10     | N/A  | 2     |
| Oracle DBA   | Degree  | DBA     | Years   | Years  |      | Years |
|              |         |         | with    | 2 with |      |       |
|              | In CS   |         |         | repli  |      | N/A   |
|              |         |         | Oracle  | cation |      |       |
+--------------+---------+---------+---------+--------+------+-------+
| Oracle DBA   | BS      | Oracle  | 5 Years | 5      | N/A  | 1     |
|              | Degree  | DBA     | with    | Years  |      | Years |
|              |         |         |         |        |      |       |
|              |         |         | Oracle  | 1 with |      | N/A   |
|              |         |         |         | repli  |      |       |
|              |         |         |         | cation |      |       |
+--------------+---------+---------+---------+--------+------+-------+
| Enterprise   | BS      | V       | 3 Years | 1 Year | 1    | 1     |
| ViewStar     | Degree  | iewStar | with    | SQL    | Year | Year  |
| A            |         |         | W       |        |      |       |
| dministrator | In CS   | In      | orkflow |        |      | 1     |
|              |         | ternals |         |        |      | Year  |
|              |         | &       |         |        |      |       |
|              |         |         |         |        |      |       |
|              |         | De      |         |        |      |       |
|              |         | veloper |         |        |      |       |
+--------------+---------+---------+---------+--------+------+-------+
| ViewStar     | Ass     | V       | 2 Year  | N/A    | 1    | 1     |
| A            | ociates | iewStar | with    |        | Year | Year  |
| dministrator |         |         | W       |        |      |       |
|              | Degree  | Admin   | orkflow |        |      | 1     |
|              |         |         |         |        |      | Year  |
+--------------+---------+---------+---------+--------+------+-------+
| Enterprise   | Ass     | Oracle  | 3 Years | 1 Year | N/A  | 1     |
| WWW          | ociates | WEB     | WEB     | PSQL   |      | Year  |
| Webmaster    | Degree  | Server  | Develop |        |      |       |
|              |         |         |         |        |      | N/A   |
+--------------+---------+---------+---------+--------+------+-------+
| Senior MS    | BS      | MCSE    | 10      | 1 Year | N/A  | 3     |
| NT/Novell    | Degree  | and/or  | Years   |        |      | Years |
| NetWare      |         |         | in PC   |        |      |       |
| Enterprise   | In CS   | CNE     | Net     |        |      | 5     |
| A            |         |         | working |        |      | Years |
| dministrator |         |         |         |        |      |       |
+--------------+---------+---------+---------+--------+------+-------+
| MS NT/Novell | BS      | MCSE    | 5 Years | N/A    | N/A  | 2     |
| NetWare      | Degree  |         | in PC   |        |      | Years |
| Enterprise   |         | And/or  | Net     |        |      |       |
| A            | In CS   |         | working |        |      | 3     |
| dministrator |         | CNE     |         |        |      | Years |
+--------------+---------+---------+---------+--------+------+-------+
| Enterprise   | BS      | Cisco   | 10      | 1 Year | N/A  | 5     |
| WAN          | Degree  | Ce      | Years   |        |      | Years |
| A            |         | rtified | in WAN  | Oracle |      | T     |
| dministrator | In CS   |         |         | Over   |      | CP/IP |
|              |         |         | 5       | WAN    |      |       |
|              |         |         | w/Cisco |        |      | 6     |
|              |         |         |         |        |      | Years |
|              |         |         |         |        |      | IP    |
|              |         |         |         |        |      | X/SPX |
+--------------+---------+---------+---------+--------+------+-------+
| LAN/WAN      | BS      | Cisco   | 7 Years | N/A    | N/A  | 3     |
| A            | Degree  | Ce      | in      |        |      | Years |
| dministrator |         | rtified | LAN/WAN |        |      | T     |
|              |         |         | Pr      |        |      | CP/IP |
|              |         |         | otocols |        |      |       |
|              |         |         |         |        |      | 4     |
|              |         |         |         |        |      | Years |
|              |         |         |         |        |      | IP    |
|              |         |         |         |        |      | X/SPX |
+--------------+---------+---------+---------+--------+------+-------+
| Helpdesk     | BS      | H       | 4 Years | 1 Year | 1    | 2     |
| Manager      | Degree  | elpdesk | H       | Gen    | Year | Years |
|              |         | C       | elpdesk | S      | Gen  |       |
|              |         | ustomer |         | upport | Sup  | Gen   |
|              |         | Srv     | 1 Year  |        | port | Su    |
|              |         | T       | Mgmt    |        |      | pport |
|              |         | raining |         |        |      |       |
+--------------+---------+---------+---------+--------+------+-------+
| National     | Ass     | H       | 2 Years | 1 Year | 1    | 1     |
| Helpdesk     | ociates | elpdesk | H       | Gen    | Year | Year  |
| Technician   |         | T       | elpdesk | S      | Gen  | Gen   |
|              | Degree  | raining |         | upport | Sup  | Su    |
|              |         |         |         |        | port | pport |
+--------------+---------+---------+---------+--------+------+-------+
| FEMA         | Ass     | MS Apps | 2 Years | 1 Year | 1    | 2     |
| Application  | ociates | Expert  | w/FEMA  | a Plus | Year | Years |
| Specialist   |         | Cert.   | Std     |        | a    |       |
|              | Degree  |         | Apps    |        | Plus |       |
|              |         | NEMIS   |         |        |      |       |
|              |         | Client  |         |        |      |       |
+--------------+---------+---------+---------+--------+------+-------+
| NEMIS System | BS      | C       | 1 Years | 1 Year | N/A  | 1     |
| Security     | Degree  | omputer | C       |        |      | Year  |
| Support      |         | & Info  | omputer | Se     |      | Sec   |
|              |         | S       | S       | curity |      | urity |
|              |         | ecurity | ecurity |        |      |       |
+--------------+---------+---------+---------+--------+------+-------+
| NEMIS        | BS      | Config  | 1 Years | N/A    | N/A  | N/A   |
| C            | Degree  | Mgmt    | Config  |        |      |       |
| onfiguration |         |         | Mgmt    |        |      |       |
| Control      |         |         |         |        |      |       |
| Support      |         |         |         |        |      |       |
+--------------+---------+---------+---------+--------+------+-------+

[]{#__RefHeading___Toc419855775 .anchor}Table 4‑1 Centralized NEMIS
Skill Set Requirement Matrix

+--------------+---------+---------+---------+--------+------+-------+
| Tier One or  | R       |         |         |        |      |       |
| Two POSITION | elative |         |         |        |      |       |
|              | E       |         |         |        |      |       |
|              | ducatio |         |         |        |      |       |
|              | n/Train |         |         |        |      |       |
|              | ing/Exp |         |         |        |      |       |
|              | erience |         |         |        |      |       |
|              | (Loc    |         |         |        |      |       |
|              | alized) |         |         |        |      |       |
+==============+=========+=========+=========+========+======+=======+
|              | Formal  | R       | R       | Oracle | VIEW | NT/NW |
|              | Ed      | elative | elative | Exp    | STAR |       |
|              |         |         | Exp     |        |      | Exp   |
|              |         | T       |         |        | Exp  |       |
|              |         | raining |         |        |      |       |
+--------------+---------+---------+---------+--------+------+-------+
| Oracle       | Ass     | Oracle  | 1 Years | 1 Year | N/A  | 1     |
| Technician   | ociates | DBA a   | with    |        |      | Years |
|              | Degree  | Plus    |         |        |      |       |
|              |         |         | Da      |        |      | N/A   |
|              |         |         | tabases |        |      |       |
+--------------+---------+---------+---------+--------+------+-------+
| ViewStar     | Ass     | V       | N/A     | N/A    | N/A  | 1     |
| Technician   | ociates | iewStar |         |        |      | Year  |
|              |         |         |         |        |      |       |
|              | Degree  | Admin a |         |        |      | 1     |
|              |         | Plus    |         |        |      | Year  |
+--------------+---------+---------+---------+--------+------+-------+
| MS NT/Novell | Ass     | NT or   | 3 Years | N/A    | N/A  | 1     |
| NetWare      | ociates | NetWare | in PC   |        |      | Year  |
| A            |         | Admin   | Net     |        |      |       |
| dministrator | Degree  |         | working |        |      | 2     |
|              |         |         |         |        |      | Years |
+--------------+---------+---------+---------+--------+------+-------+
| LAN          | Ass     | LAN     | 2 Years | N/A    | N/A  | 1     |
| A            | ociates | Admin   | in      |        |      | Year  |
| dministrator |         | (pro    | Pr      |        |      | T     |
|              | Degree  | tocols) | otocols |        |      | CP/IP |
|              |         |         |         |        |      |       |
|              |         |         |         |        |      | 2     |
|              |         |         |         |        |      | Years |
|              |         |         |         |        |      | IP    |
|              |         |         |         |        |      | X/SPX |
+--------------+---------+---------+---------+--------+------+-------+
| Helpdesk     | Ass     | H       | 1 Years | N/A    | N/A  | 2     |
| Manager      | ociates | elpdesk | H       |        |      | Years |
|              | Degree  | C       | elpdesk |        |      |       |
|              |         | ustomer |         |        |      | Gen   |
|              |         | Srv     | 1 Year  |        |      | Su    |
|              |         | T       | Mgmt    |        |      | pport |
|              |         | raining |         |        |      |       |
+--------------+---------+---------+---------+--------+------+-------+
| Helpdesk     | Ass     | H       | 2 Years | N/A    | N/A  | 1     |
| Technician   | ociates | elpdesk | H       |        |      | Year  |
|              |         | T       | elpdesk |        |      | Gen   |
|              | Degree  | raining |         |        |      | Su    |
|              |         |         |         |        |      | pport |
+--------------+---------+---------+---------+--------+------+-------+
| PC           | Ass     | A+ PC   | 2 Years | N/A    | N/A  | 1     |
| Technician   | ociates | Repair  | He      |        |      | Year  |
|              |         | T       | lpdesk/ |        |      | Gen   |
|              | Degree  | raining |         |        |      | Su    |
|              |         |         | PC      |        |      | pport |
|              |         |         | Repair  |        |      |       |
+--------------+---------+---------+---------+--------+------+-------+
| FEMA         | Ass     | MS Apps | 2 Years | N/A    | N/A  | 1     |
| Application  | ociates | Expert  | w/FEMA  |        |      | Year  |
| Specialist   |         | Cert.   | Std     |        |      | a     |
|              | Degree  |         | Apps    |        |      | Plus  |
|              |         | NEMIS   |         |        |      |       |
|              |         | Client  |         |        |      |       |
+--------------+---------+---------+---------+--------+------+-------+

[]{#__RefHeading___Toc419855776 .anchor}Table 4‑2 Localized NEMIS Skill
Set Requirement Matrix

# Site Staffing Requirements

NEMIS provides FEMA with the unique opportunity to implement a
functional software solution to what is currently accomplished primarily
via manual/hardcopy (i.e., pencil, paper, FAX, e-mail). While this is a
benefit to the end user, it poses several support issues to FEMA O&M.
First and foremost, how do you staff such a system?

Staffing an existing system is a very difficult task. On several
occasions over the past few years, FEMA has hired independent
contractors that specialize in evaluating their clients existing
staffing and provided recommendations for improving the staffing
structure. To say the least, identifying personnel, their skill set
requirements, and staffing locations for NEMIS is a very complicated
task.

Our approach for identifying site staffing requirements was to discuss
the environment with the developers, industry experts, and NEMIS
specialists, provide them with an overview of the operational approach,
and request their input for staffing requirements. Staffing requirements
for Helpdesk functions were the most difficult to identify. This is due
to the fact that NEMIS has no benchmark for Helpdesk statistics. The
industry uses Helpdesk and Helpdesk technician availability statistics
to justify staffing requirements. Therefore, we used an estimate based
on a ratio of number of technicians to number of users. Our estimate
uses a ratio of one Helpdesk technician (telephone support) to every 200
users, one PC technician (hardware support) to every 100 users, and one
Application specialist (software support) to every 100 users. The tables
shown below are the results of this analysis.

## Centralized Staffing Requirement

The table below shows the estimated staffing requirements for supporting
NEMIS at Mount Weather and using HQ FEMA as a backup site. The table
also attempts to identify on-hand resources and provide an estimated
staffing short fall. This effort is a long and tedious undertaking and
is currently underway. The document will be updated as personnel are
identified and assigned to positions.

+------------------+---------+-------+---------+------+------+------+
| Centralized      | Mount   | HQ    | P       | On-  | On-  | S    |
|                  | weather | FEMA  | osition | Hand | Hand | hort |
| position         |         |       | Total   |      |      | fall |
|                  |         |       |         | FTE  | C    |      |
|                  |         |       |         |      | ont. |      |
+==================+=========+=======+=========+======+======+======+
| Tier 3, Lvl 2 -- | 2       | 1     | 3       | 0    | 0    | 3    |
| Senior           |         |       |         |      |      |      |
|                  |         |       |         |      |      |      |
| Enterprise       |         |       |         |      |      |      |
| Oracle DBA       |         |       |         |      |      |      |
+------------------+---------+-------+---------+------+------+------+
| Tier 3, Lvl 2 -- | 2       | 1     | 3       | 0    | 0    | 3    |
| Enterprise       |         |       |         |      |      |      |
| Oracle DBA       |         |       |         |      |      |      |
| Support          |         |       |         |      |      |      |
+------------------+---------+-------+---------+------+------+------+
| Tier 3, Lvl 2 -- | 2       | 1     | 3       | 0    | 0    | 3    |
| Senior           |         |       |         |      |      |      |
|                  |         |       |         |      |      |      |
| Enterprise       |         |       |         |      |      |      |
| ViewStar Admin   |         |       |         |      |      |      |
+------------------+---------+-------+---------+------+------+------+
| Tier 3, Lvl 2 -- | 1       | 1     | 2       | 0    | 0    | 2    |
| Enterprise WWW   |         |       |         |      |      |      |
| Webmaster        |         |       |         |      |      |      |
+------------------+---------+-------+---------+------+------+------+
| Tier 3, Lvl 2 -- | 2       | 1     | 3       | 0    | 0    | 3    |
|                  |         |       |         |      |      |      |
| Senior           |         |       |         |      |      |      |
| Enterprise       |         |       |         |      |      |      |
| Server/LAN Admin |         |       |         |      |      |      |
+------------------+---------+-------+---------+------+------+------+
| Tier 3, Lvl 2 -- | 2       | 1     | 3       | 0    | 0    | 3    |
|                  |         |       |         |      |      |      |
| Enterprise       |         |       |         |      |      |      |
| LAN/Server Admin |         |       |         |      |      |      |
| Support          |         |       |         |      |      |      |
+------------------+---------+-------+---------+------+------+------+
| Tier 3, Lvl 2 -- | 2       | 1     | 3       | 0    | 0    | 3    |
| Senior           |         |       |         |      |      |      |
| Enterprise WAN   |         |       |         |      |      |      |
| Admin            |         |       |         |      |      |      |
+------------------+---------+-------+---------+------+------+------+
| Tier 3, Lvl 2 -- | 2       | 1     | 3       | 0    | 0    | 3    |
| Enterprise WAN   |         |       |         |      |      |      |
| Admin Support    |         |       |         |      |      |      |
+------------------+---------+-------+---------+------+------+------+
| Tier 3, Lvl 2 -- | 1       | 1     | 2       | 0    | 0    | 2    |
| Enterprise       |         |       |         |      |      |      |
| Config Mgmt      |         |       |         |      |      |      |
| Support          |         |       |         |      |      |      |
+------------------+---------+-------+---------+------+------+------+
| Tier 3, Lvl 2 -- | 1       | 1     | 2       | 0    | 0    | 2    |
| Enterprise       |         |       |         |      |      |      |
| Security Support |         |       |         |      |      |      |
+------------------+---------+-------+---------+------+------+------+
| Tier 3, Lvl 2 -- | 4       | 2     | 6       | 0    | 0    | 6    |
| Enterprise       |         |       |         |      |      |      |
| Application      |         |       |         |      |      |      |
| Support          |         |       |         |      |      |      |
+------------------+---------+-------+---------+------+------+------+
| Tier 3, Lvl 1 -- | 6       | 4     | 10      | 0    | 0    | 10   |
| National         |         |       |         |      |      |      |
| Helpdesk Support |         |       |         |      |      |      |
+------------------+---------+-------+---------+------+------+------+
| SITE TOTALS      | 27      | 16    | 43      | 0    | 0    | 43   |
+------------------+---------+-------+---------+------+------+------+

[]{#__RefHeading___Toc419855777 .anchor}Table 5.1‑3 Estimated NEMIS
Centralized Staffing Requirements

## Localized Fixed Facility Staffing Requirements

Table 5.2-4 Estimated NEMIS Localized Fixed Site Staffing Requirements
shows the estimated staffing requirements for the large FEMA fixed
sites. Note that the Mount Weather site represents support for the local
user community and are above and beyond the centralized staffing
requirement shown above. The figures shown are based upon the following
assumptions.

-   400 Mt Weather Users

-   800 HQ FEMA Users

-   640 TX NPSC/TX Users

-   500 MD NPSC/NTC Users

-   244 VA NPSC/NTC Users

This effort is also underway and the document will be updated
periodically to show staffing assignment progress.

<table>
<colgroup>
<col style="width: 13%" />
<col style="width: 12%" />
<col style="width: 8%" />
<col style="width: 8%" />
<col style="width: 8%" />
<col style="width: 8%" />
<col style="width: 11%" />
<col style="width: 11%" />
<col style="width: 8%" />
<col style="width: 10%" />
</colgroup>
<thead>
<tr class="header">
<th><p>Local</p>
<p>position</p></th>
<th>Mount weather</th>
<th>HQ FEMA</th>
<th>TX NPSC /NTC</th>
<th>MD NPSC /NTC</th>
<th><p>Va</p>
<p>npsc /NTC</p></th>
<th>Totals</th>
<th><p>On-Hand</p>
<p>FTE/DTE</p></th>
<th><p>On-Hand</p>
<p>Cont</p></th>
<th>Short fall</th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td>Tier 2 –Oracle Technician</td>
<td>2.5</td>
<td>.5</td>
<td>.5</td>
<td>.5</td>
<td>.5</td>
<td>4.5</td>
<td>0</td>
<td>0</td>
<td>4.5</td>
</tr>
<tr class="even">
<td>Tier 2 –ViewStar Technician</td>
<td>.5</td>
<td>.5</td>
<td>.5</td>
<td>.5</td>
<td>.5</td>
<td>2.5</td>
<td>0</td>
<td>0</td>
<td>2.5</td>
</tr>
<tr class="odd">
<td><p>Tier 2 -</p>
<p>WAN Support</p></td>
<td>2</td>
<td>2</td>
<td>2</td>
<td>1</td>
<td>1</td>
<td>8</td>
<td>0</td>
<td>0</td>
<td>8</td>
</tr>
<tr class="even">
<td><p>Tier 2 – Server/LAN</p>
<p>Admin</p></td>
<td>2</td>
<td>2</td>
<td>3</td>
<td>2</td>
<td>2</td>
<td>11</td>
<td>0</td>
<td>0</td>
<td>11</td>
</tr>
<tr class="odd">
<td>Helpdesk Manager</td>
<td>1</td>
<td>1</td>
<td>2</td>
<td>1</td>
<td>1</td>
<td>6</td>
<td>0</td>
<td>0</td>
<td>6</td>
</tr>
<tr class="even">
<td>Helpdesk Technician</td>
<td>6</td>
<td>4</td>
<td>3.5</td>
<td>2.5</td>
<td>1.5</td>
<td>17.5</td>
<td>0</td>
<td>0</td>
<td>17.5</td>
</tr>
<tr class="odd">
<td>PC Technician</td>
<td>4</td>
<td>8</td>
<td>6.5</td>
<td>5</td>
<td>2.5</td>
<td>26</td>
<td>0</td>
<td>0</td>
<td>26</td>
</tr>
<tr class="even">
<td>Application Technician</td>
<td>4</td>
<td>8</td>
<td>6.5</td>
<td>5</td>
<td>2.5</td>
<td>26</td>
<td>0</td>
<td>0</td>
<td>26</td>
</tr>
<tr class="odd">
<td>SITE TOTALS</td>
<td>22</td>
<td>26</td>
<td>24.5</td>
<td>17.5</td>
<td>11.5</td>
<td>101.5</td>
<td>0</td>
<td>0</td>
<td>101.5</td>
</tr>
</tbody>
</table>

[]{#__RefHeading___Toc419855778 .anchor}Table 5.2‑4 Estimated NEMIS
Localized Fixed Site Staffing Requirements

### Mount Weather Staffing and Skill Set Requirements

  ------------------------------------------------------------------------
  position                             Personnel    Requirement
  ------------------------------------ ------------ ----------------------
  Senior Enterprise Oracle DBA         2            Central/Enterprise
                                                    Systems Management
                                                    Team

  Enterprise Oracle DBA Support        2            Central Management

  Senior Enterprise ViewStar Admin     2            Central/Enterprise
                                                    Systems Management
                                                    Team

  Senior Enterprise Server/LAN Admin   2            Central/Enterprise
                                                    Systems Management
                                                    Team

  Senior Enterprise WAN Admin          2            Central/Enterprise
                                                    Systems Management
                                                    Team

  Enterprise WAN Admin Support         2            Central Management

  NEMIS System Security Manager        1            Central/Enterprise
                                                    Systems Management
                                                    Team

  NEMIS Config Control Manager         1            Central/Enterprise
                                                    Systems Management
                                                    Team

                                       14           
  ------------------------------------------------------------------------

[]{#__RefHeading___Toc419855779 .anchor}Table 5.2.1‑5 Centralized
Enterprise Staffing Requirements (Mount Weather)

  ------------------------------------------------------------------------
  position                        Personnel    Requirement
  ------------------------------- ------------ ---------------------------
  Oracle Administrator            .5           MWEAC Site Support

  ViewStar Administrator          .5           MWEAC Site Support

  LAN/WAN Administrator           2            

  Server LAN Administrator        2            MWEAC Site Support

  Helpdesk Manager                1            Enterprise/MWEAC Site
                                               Support

  Helpdesk Technicians            6            Enterprise/MWEAC Site
                                               Support

  PC Technicians                  4            Enterprise/MWEAC Site
                                               Support

  Application Specialists         4            Enterprise/MWEAC Site
                                               Support

                                  20           
  ------------------------------------------------------------------------

[]{#__RefHeading___Toc419855780 .anchor}Table 5.2.1‑6 MWEAC Site
Staffing Requirements

  ------------------------------------------------------------------------
  position                        Personnel    Requirement
  ------------------------------- ------------ ---------------------------
  Oracle Administrator            .5           Virginia NPSC

  ViewStar Administrator          .5           Virginia NPSC

  MS NT/NetWare Server            2            Virginia NPSC
  Administrator                                

  LAN/WAN Administrator           1            Virginia NPSC

  Helpdesk Manager                1            Virginia NPSC

  Helpdesk Technicians            1.5          Virginia NPSC

  PC Technicians                  2.5          Virginia NPSC

  Application Specialists         2.5          Virginia NPSC

                                  11.5         
  ------------------------------------------------------------------------

[]{#__RefHeading___Toc419855781 .anchor}Table 5.2.1‑7 Virginia NPSC/NTC
Staffing Requirements

### Denton Staffing Requirements

  ------------------------------------------------------------------------
  position                        Personnel    Requirement
  ------------------------------- ------------ ---------------------------
  Server/LAN Administrator        1            Denton NTC

  LAN/WAN Administrator           1            Denton NTC

  Helpdesk Manager                1            Denton NTC

  Helpdesk Technicians            1.5          Denton NTC

  PC Technicians                  2.5          Denton NTC

  Application Specialists         2.5          Denton NTC

                                  9.5          
  ------------------------------------------------------------------------

[]{#__RefHeading___Toc419855782 .anchor}Table 5.2.2‑8 Denton NTC
Staffing Requirements

  ------------------------------------------------------------------------
  position                        Personnel    Requirement
  ------------------------------- ------------ ---------------------------
  Oracle Administrator            .5           Denton NPSC

  ViewStar Administrator          .5           Denton NPSC

  Server/LAN Administrator        2            Denton NPSC

  LAN/WAN Administrator           1            Denton NPSC

  Helpdesk Manager                1            Denton NPSC

  Helpdesk Technicians            2            Denton NPSC

  PC Technicians                  4            Denton NPSC

  Application Specialists         4            Denton NPSC

                                  15           
  ------------------------------------------------------------------------

[]{#__RefHeading___Toc419855783 .anchor}Table 5.2.2‑9 Denton NPSC
Staffing Requirements

### Hyattsville NPSC/NTC Staffing Requirements

  ------------------------------------------------------------------------
  position                        Personnel    Requirement
  ------------------------------- ------------ ---------------------------
  Oracle Administrator            .5           Hyattsville NPSC/NTC

  ViewStar Administrator          .5           Hyattsville NPSC/NTC

  MS NT/NetWare Server            2            Hyattsville NPSC/NTC
  Administrator                                

  LAN/WAN Administrator           1            Hyattsville NPSC/NTC

  Helpdesk Manager                1            Hyattsville NPSC/NTC

  Helpdesk Technicians            2.5          Hyattsville NPSC/NTC

  PC Technicians                  5            Hyattsville NPSC/NTC

  Application Specialists         5            Hyattsville NPSC/NTC

                                  17.5         
  ------------------------------------------------------------------------

[]{#__RefHeading___Toc419855784 .anchor}Table 5.2.3‑10 Hyattsville
NPSC/NTC Staffing Requirements

### FEMA HQ Staffing Requirements

  ------------------------------------------------------------------------
  position                        Personnel    Requirement
  ------------------------------- ------------ ---------------------------
  Oracle Report Writer            1            FEMA HQ

  Oracle Administrator            .5           FEMA HQ

  ViewStar Administrator          .5           FEMA HQ

  NEMIS System Security Manager   1            Central Management

  MS NT/NetWare Server            2            FEMA HQ
  Administrator                                

  NEMIS WEB Master                2            Central/Enterprise Systems
                                               Management Team

  LAN/WAN Administrator           2            FEMA HQ

  Helpdesk Manager                1            FEMA HQ

  Helpdesk Technicians            4            FEMA HQ

  PC Technicians                  8            FEMA HQ

  Application Specialists         8            FEMA HQ

                                  30           
  ------------------------------------------------------------------------

[]{#__RefHeading___Toc419855785 .anchor}Table 5.2.4‑11 HQ Site Staffing
Requirements

## Regional Staffing Requirements

NEMIS must be supported at all locations. Therefore, adequate staff must
be identified to maintain the system in the Regions. The tables below
attempt to identify staffing levels associated with the predefined Tier
One and Two positions. The figures shown make the following assumptions.

-   One Helpdesk technician (telephone support) to every 200 users;

-   One PC technician (hardware support) to every 100 users; and

-   One application specialist (software support) to every 100 users.

Furthermore, given the complexity of the NEMIS system and subsystems, no
site will have less than 3 personnel available to support NEMIS.
Identifying support staff for the Regional sites requires specific
knowledge of the site and its users. Over the next several months, O&M
management will be working with the Regional representatives to identify
their site requirements and assist them in their efforts to support the
NEMIS environment. This document will be updated with specific Regional
staffing figures as they are identified.

  ------------------------------------------------------------------------
  position                        Personnel    Requirement
  ------------------------------- ------------ ---------------------------
  Oracle Administrator            .5           Region I

  ViewStar Administrator          .5           Region I

  MS NT/NetWare Server            1            Region I
  Administrator                                

  LAN/WAN Administrator           .5           Region I

  Helpdesk Manager                1            Region I

  Helpdesk Technicians            TBD          Region I

  PC Technicians                  TBD          Region I

  Application Specialists         TBD          Region I

  Totals                          TBD          
  ------------------------------------------------------------------------

[]{#__RefHeading___Toc419855786 .anchor}Table 5.3‑12 Region I Staffing
Requirements

  ------------------------------------------------------------------------
  position                        Personnel    Requirement
  ------------------------------- ------------ ---------------------------
  Oracle Administrator            .5           Region II

  ViewStar Administrator          .5           Region II

  MS NT/NetWare Server            1            Region II
  Administrator                                

  LAN/WAN Administrator           .5           Region II

  Helpdesk Manager                1            Region II

  Helpdesk Technicians            TBD          Region II

  PC Technicians                  TBD          Region II

  Application Specialists         TBD          Region II

  Totals                          TBD          
  ------------------------------------------------------------------------

[]{#__RefHeading___Toc419855787 .anchor}Table 5.3‑13 Region II Staffing
Requirements

  ------------------------------------------------------------------------
  position                        Personnel    Requirement
  ------------------------------- ------------ ---------------------------
  Oracle Administrator            .5           Region III

  ViewStar Administrator          .5           Region III

  MS NT/NetWare Server            1            Region III
  Administrator                                

  LAN/WAN Administrator           .5           Region III

  Helpdesk Manager                1            Region III

  Helpdesk Technicians            TBD          Region III

  PC Technicians                  TBD          Region III

  Application Specialists         TBD          Region III

  Totals                          TBD          
  ------------------------------------------------------------------------

[]{#__RefHeading___Toc419855788 .anchor}Table 5.3‑14 Region III Staffing
Requirements

  ------------------------------------------------------------------------
  position                        Personnel    Requirement
  ------------------------------- ------------ ---------------------------
  Oracle Administrator            .5           Region IV

  ViewStar Administrator          .5           Region IV

  MS NT/NetWare Server            1            Region IV
  Administrator                                

  LAN/WAN Administrator           .5           Region IV

  Helpdesk Manager                1            Region IV

  Helpdesk Technicians            TBD          Region IV

  PC Technicians                  TBD          Region IV

  Application Specialists         TBD          Region IV

  Totals                          TBD          
  ------------------------------------------------------------------------

[]{#__RefHeading___Toc419855789 .anchor}Table 5.3‑15 Region IV Staffing
Requirements

  ------------------------------------------------------------------------
  position                        Personnel    Requirement
  ------------------------------- ------------ ---------------------------
  Oracle Administrator            .5           Region V

  ViewStar Administrator          .5           Region V

  MS NT/NetWare Server            1            Region V
  Administrator                                

  LAN/WAN Administrator           .5           Region V

  Helpdesk Manager                1            Region V

  Helpdesk Technicians            TBD          Region V

  PC Technicians                  TBD          Region V

  Application Specialists         TBD          Region V

  Totals                          TBD          
  ------------------------------------------------------------------------

[]{#__RefHeading___Toc419855790 .anchor}Table 5.3‑16 Region V Staffing
Requirements

  ------------------------------------------------------------------------
  position                        Personnel    Requirement
  ------------------------------- ------------ ---------------------------
  Oracle Administrator            .5           Region VI

  ViewStar Administrator          .5           Region VI

  MS NT/NetWare Server            1            Region VI
  Administrator                                

  LAN/WAN Administrator           .5           Region VI

  Helpdesk Manager                1            Region VI

  Helpdesk Technicians            TBD          Region VI

  PC Technicians                  TBD          Region VI

  Application Specialists         TBD          Region VI

  Totals                          TBD          
  ------------------------------------------------------------------------

[]{#__RefHeading___Toc419855791 .anchor}Table 5.3‑17 Region VI Staffing
Requirements

  ------------------------------------------------------------------------
  position                        Personnel    Requirement
  ------------------------------- ------------ ---------------------------
  Oracle Administrator            .5           Region VII

  ViewStar Administrator          .5           Region VII

  MS NT/NetWare Server            1            Region VII
  Administrator                                

  LAN/WAN Administrator           .5           Region VII

  Helpdesk Manager                1            Region VII

  Helpdesk Technicians            TBD          Region VII

  PC Technicians                  TBD          Region VII

  Application Specialists         TBD          Region VII

  Totals                          TBD          
  ------------------------------------------------------------------------

[]{#__RefHeading___Toc419855792 .anchor}Table 5.3‑18 Region VII Staffing
Requirements

  ------------------------------------------------------------------------
  position                        Personnel    Requirement
  ------------------------------- ------------ ---------------------------
  Oracle Administrator            .5           Region VIII

  ViewStar Administrator          .5           Region VIII

  MS NT/NetWare Server            1            Region VIII
  Administrator                                

  LAN/WAN Administrator           .5           Region VIII

  Helpdesk Manager                1            Region VIII

  Helpdesk Technicians            TBD          Region VIII

  PC Technicians                  TBD          Region VIII

  Application Specialists         TBD          Region VIII

  Totals                          TBD          
  ------------------------------------------------------------------------

[]{#__RefHeading___Toc419855793 .anchor}Table 5.3‑19 Region VIII
Staffing Requirements

  ------------------------------------------------------------------------
  position                        Personnel    Requirement
  ------------------------------- ------------ ---------------------------
  Oracle Administrator            .5           Region IX

  ViewStar Administrator          .5           Region IX

  MS NT/NetWare Server            1            Region IX
  Administrator                                

  LAN/WAN Administrator           .5           Region IX

  Helpdesk Manager                1            Region IX

  Helpdesk Technicians            TBD          Region IX

  PC Technicians                  TBD          Region IX

  Application Specialists         TBD          Region IX

  Totals                          TBD          
  ------------------------------------------------------------------------

[]{#__RefHeading___Toc419855794 .anchor}Table 5.3‑20 Region IX Staffing
Requirements

  ------------------------------------------------------------------------
  position                        Personnel    Requirement
  ------------------------------- ------------ ---------------------------
  Oracle Administrator            .5           Region X

  ViewStar Administrator          .5           Region X

  MS NT/NetWare Server            1            Region X
  Administrator                                

  LAN/WAN Administrator           .5           Region X

  Helpdesk Manager                1            Region X

  Helpdesk Technicians            TBD          Region X

  PC Technicians                  TBD          Region X

  Application Specialists         TBD          Region X

  Totals                          TBD          
  ------------------------------------------------------------------------

[]{#__RefHeading___Toc419855795 .anchor}Table 5.3‑21 Region X Staffing
Requirements

# Maintenance

O&M has the charter to ensure that the NEMIS environment operate
smoothly with as little down time as possible. While the design of the
system includes fault tolerant features like UPS, multiple CPU servers,
and redundant power supplies, there is still a chance that the system or
a system component may fail. These situations must be planned for to
mitigate extended down time of the server.

Furthermore, all devices have a "useful" life cycle. If the device
becomes antiquated or breaks, it must be replaced. O&M must monitor the
NEMIS environment and plan for stocking spare parts and the ultimate
replacement of the equipment. Monitoring the server environment will
provide historical trend information to justify the expansion of the
hard disk or memory systems. Consistent monitoring of the system will
also identify component failure trends and allow FEMA O&M to plan the
long term funding of spare parts.

Monitoring the system and tracking trend data will also aid in the
planning for long term funding of the life cycle. As parts are used they
must be replaced. As systems become antiquated or break, new systems
must be purchased. All of these efforts combined make up the life cycle
maintenance of the NEMIS environment.

#   {#section .unnumbered}

# Assumptions and Constraints

The staffing figures include FEMA full-time employees, Disaster
Temporary Employees, and contractors. It is assumed that FEMA does not
currently employ individuals with the necessary skills and background to
staff the NEMIS Enterprise Systems Management Team and that this
requirement will be out-sourced.

The approach for implementing the support structure must not only
include capable trained personnel, but also a self supporting life cycle
program that keeps people trained as individuals cycle in and out of the
environment. This can be accomplished via formal classroom training
and/or On-The-Job Training (OJT). The O&M NEMIS Training Plan must be
developed and it must identify a training structure that supports long
term stability.

The three levels of support require O&M guidelines and standards in
order to function as a single support system. It is assumed that SOPs
and policies for maintaining the NEMIS environment will be developed and
updated as required.

This document assumes that the staff tasked to maintain the NEMIS
environment will be provided with adequate tools (hardware and software)
to accomplish their day-to-day duties.

While there is some overlap with non-NEMIS O&M system support, this
document was developed to identify an O&M concept for managing NEMIS. If
this document satisfies FEMA's requirements for the O&M of NEMIS and can
be supported and implemented, it could be used as a model for the O&M of
other non-NEMIS activities.

Appendix A

# Proposed BMC Architecture for NEMIS

BMC Patrol will be used to monitor the primary components of the NEMIS
environment; MS NT, Oracle RDBMS, MS Exchange, ViewStar, and the NEMIS
Human Services subsystem. This will be accomplished by loading the BMC
Patrol Agent on all NEMIS servers that run one, some, or all of these
primary components. The Agent runs as a service in the NT environment
and collects data on specific aspects of the environment at
predetermined intervals. The BMC Patrol Administrators control the
specific components monitored. The Administrators also control the
interval at which the Agent polls the component. The data collected by
the Agents is stored locally on the NT server. Periodically the data
will be extracted and stored in a centralized data repository for
historical system research.

![](media/image10.wmf)

Local administrators will use the PatrolWatch tool to monitor the
over-all health of the NEMIS servers. This tool operates much the same
way as the Console, but lacks the Agent configuration capability of the
centralized tool. Figure A-2 presents an overview of the architecture
described above. Both the central and local tools will notify the
administrators of a problem by flashing the offending site icon. The
administrators can then drill down through the site and server icons to
the specific component.

Appendix B

# Proposed Tivoli Architecture for NEMIS

In order to facilitate simplified and secure user management in the
NEMIS[^1] environment, the TME 10 Framework and TME 10 User
Administration products will be deployed nationwide on all NEMIS
systems. TME 10 will provide a simple, easy to use environment for
administrators to manage user accounts. All actions taken by subordinate
administrators are controlled and enforced by policy defined by senior
NEMIS management. This paradigm allows for the secure delegation of
administrative tasks to junior administrators without the need for
constant oversight, the time for which will be at a premium when the
organization is operating in crisis mode.

The TMR[^2] layout will mirror the anticipated layout of the Windows NT
Domain structure within NEMIS. This will provide a clear and consistent
environment for the administrative staff and minimize ambiguities that
could compromise security and otherwise hinder disaster assistance
personnel in the performance of their assigned duties.

TMR Design

The TMR server is the heart of the Tivoli Management Environment (TME).
All managed resources must be defined to a TMR server. All of the
desired Tivoli management applications must also be installed on the TMR
servers (some may also have binaries that must be installed on the
managed devices as well). The TMR server serves as the central arbiter
within the systems management strategy that allows all of the various
applications and components to communicate and integrate with each
other.

Server and Managed Nodes

The TMR will consist of the TMR Server, located at the Mt. Weather, VA
facility. The hardware for this system will be one of the NEMIS systems
already in existence. It is anticipated that the addition of TMR duties
will not adversely affect this machine in any way. The TME 10 software
and database will require approximately 300 MB of disk storage.

The TMR also will consist of approximately 74 Managed Nodes[^3]. These
systems are made up of the NEMIS applications servers, Oracle database
servers, etc. These systems are also Compaq SMP systems. As in the case
of the TMR server, it is not anticipated that the TME software will in
any way adversely effect NEMIS operations from a system resources point
of view.

Conceptually, the TMR can be visualized in Figure b-9. As can be seen,
the TMR forms a sort of virtual network of Tivoli managed systems.
Managed nodes are not constrained by physical location within the FEMA
Switching Network.

Policy Region Design

A Policy Region is a collection of TME 10 resources that are governed by
a common set of policies. Policies are rules that govern the management
of resources, such as saying that all user accounts must have passwords,
and password aging must be enabled. Traditionally, these rules take the
form of shell scripts, written procedures and/or guidelines, some of
which may be known only to senior system administrators. The TME 10
framework provides a multi-level policy facility that will enable NEMIS'
senior administrative staff to encode acceptable policies and values for
individual resources. This will result in the rules set forth being
strictly enforced across-the-board, allowing for the secure delegation
of system management tasks to other administrators within the
enterprise.

Policy regions will be created within the NEMIS environment to represent
a management domain or "sphere of influence" for one or more system
administrators. These policy regions will strictly enforce the policies
associated with the region and its resources. Administrators will be
delegated resource management authority over one or more of these policy
regions. They will only be allowed to perform operations and changes
that are consistent with policies that have been set up.

The policy region layout for NEMIS will closely mirror the NT domain
layout. It is anticipated there will be one policy region representing
each of the 10 FEMA regions, one for each of the three NPSCs, and one
for Headquarters in Washington D.C. There will also be one additional
policy region created to hold the standby DFO systems. This will allow
them to be as up-to-date as possible when they are deployed to a
disaster site.

Policy Region Naming Conventions

With the exception of the top-level policy region and the region holding
the inactive DFO systems, policy region names will follow the NT domain
naming convention established by FEMA when it is finalized. Until this
occurs, region names in this document will be geographic (e.g.: Region
5, Hyattsville, Headquarters, etc.)

The layout of the policy regions is illustrated in Figure b-10. It will
be a two tiered hierarchical construct centered around the "NEMIS"
top-level policy region.

Policy Region Contents

With few exceptions, the contents of the policy regions will be uniform;
differing only in the number of managed nodes assigned.

Policy regions will contain Managed Nodes, Profile Managers[^4], and
perhaps a Task Library[^5]. Managed nodes will be created by senior
administrative staff in the policy region where they reside as described
above. Each policy region will have a Profile Manager containing the
required User and Group Profiles for that region. The layout and
contents of a sample policy region is illustrated in Figure b-10.

Profile Managers and Profiles

In TME 10, all user and group account creation is done within the
context of a Profile Manager. Profile managers can contain two basic
entities: Profiles[^6] and Subscribers. For NEMIS there will be two
types of profile in use: User and Group. The icons for user and group
profiles that will appear in the profile manager are illustrated in
figures 5 and 6 respectively.

The User profile will contain information such as user name, login name,
user ID, user group, home directory, etc., for each user that is defined
in that profile. In addition, the user profiles for NEMIS will be
customized to add additional fields required to create and manage Oracle
accounts and add data to selected Oracle tables on the Consolidated
Master. Additionally, the Add User dialog screen will be modified to add
an additional button. When the administrator makes this selection, he or
she will be presented with a list of valid users drawn from the ADD
database. Group profiles will be used to create and maintain NT
administrative groups. These allow for the simplification of granting
privileges to users by grouping them administratively and granting the
privilege to the group as a whole.

Default and Validation Policies

As stated earlier in this document, policies are used throughout TME 10
to what can be done and how it is to be accomplished. TME 10 User
Administration is no exception. There are two sets of policies that come
into play.

The Default policy mechanism is engaged from the Add User dialog screen
when the administrator chooses the "Generate Defaults" button. This
allows the administrator to enter a minimum amount of data on the
initial form; the system will fill in the rest of the blanks in
accordance with the policies defined by senior NEMIS staff.

After defaults have been generated, the administrator is afforded the
opportunity to make any changes deemed necessary. When all changes are
made, the administrator selects either the "Add and Close" or "Add"
button. At this time, the Validation policy mechanism is run against the
new and/or modified user data. Validation policy ensures that the user
account is created in a way that complies with NEMIS standards. It will
not allow a non-compliant account to be created. The administrator will
have to make the necessary changes to the user data before the account
can be created. Typically, if the generated defaults are used
exclusively, then an account will never fail validation.

There can be Default and Validation policies in place for each field on
the "Add User" dialog screen. These policies are typically shell or Perl
scripts, but can be converted to binary executables if desired.

Profile Distribution

In order for the new user account to physically be created, the user
profile must be distributed to its subscribers. This operation is
accomplished in one of several ways:

-   Select "Distribute" from the "Profile" drop down menu in the "User
    Profile Properties" dialog. Another dialog will appear allowing the
    administrator to define the terms of distribution and which
    subscribers to distribute to.

-   The admin is also afforded to opportunity to "schedule" the
    distribution, and hence account creation, for some time in the
    future.

-   The administrator can "drag-and-drop" the icon for the user profile
    onto the selected subscribers.

-   Select the profiles to be distributed and the subscribers to
    distribute to then select "Distribute" from the "Profile Manager"
    drop down menu while at the Profile Manager screen.

-   From the command line: ***C:\\ wdistrib \@UserProfile:Region_5_Users
    \@ManagedNode:Region_5_PDC***

The overall architecture for NEMIS calls for NT accounts to only be
created on PDC systems. Thus, only NT PDCs will be a subscriber to the
User Accounts profiles. NT "trust relationships" will be relied upon to
ensure that the users' accounts are valid on systems where a user needs
access.

Oracle Accounts

The creation or deletion of an NT account will automatically trigger the
creation/deletion of an Oracle account for that user on all Oracle
servers throughout NEMIS.

NEMIS Accounts

The creation or deletion of an NT account will automatically trigger the
NEMIS account information in the USER_DEF table to be flagged inactive.
This action will be performed on the NEMIS master consolidated database
and Oracle replication will push the inactive user status out to all
NEMIS sites.

PadLock Accounts

The creation or deletion of an NT account will automatically trigger the
creation/deletion of the PadLock account for that user on the master
PadLock server (master consolidated database). Oracle replication will
push the deleted user out to all NEMIS sites.

Tivoli Administrators

NEMIS' senior administrators will only create Tivoli administrators for
the various policy regions. There will be two or three senior
administrators with the requisite global TMR roles (privileges).

Each policy region will have at least two Tivoli administrators with the
appropriate roles to perform the necessary User Admin functions within
their respective regions only. The senior Tivoli administrators can only
grant additional resource roles, which allow them to perform
administrative functions outside their region.

The senior administrators will have all privileges TMR wide. Regional
administrators will have only those privileges needed to effectively
perform their job.

Administrators will log in to one of their local NT systems. There they
will be validated by NTs security mechanism. After successful NT
validation, the TMR server as legitimate Tivoli administrators will
validate them again. Only then will they be granted access to their
Tivoli Desktop. Failed validations at any point result in a security
event being generated in the NT Event Log.

The administrators' Tivoli Desktop will contain icons for only those
resources over which they have administrative purview. Attempted actions
outside of defined policy will result in privilege violation errors.
These policies can only be changed or overridden by the senior
administrators.

The senior Tivoli administrators' desktops will contain icons to all
managed resources within the TMR. New administrators can only be created
from these desktops. All administrator accounts are stored within the
"Administrators" collection (iconic representation in Figure 7). Opening
this collection will give the senior administrator access to all Tivoli
administrators' accounts (Tivoli accounts...). The iconic representation
for an administrator is shown in Figure 8.

Appendix C

# Proposed SMS Architecture for NEMIS

The SMS architecture is currently under development. Therefore the
following description is one possible architecture that maybe
implemented. This document will be updated as the SMS architecture is
finalized and made available for general use. The primary use of SMS in
the NEMIS environment is software distribution. Therefore, the
architecture defined below is optimized for cost, ease of management,
and functionality for an SMS implementation used primarily for software
distribution.

NEMIS will use a single SMS environment with one primary SMS site and
several secondary SMS sites. Primary SMS servers require MS SQL servers
to provide the SMS data repository. Secondary site support primary site
distribution and monitoring. The NEMIS primary SMS site will be homed at
Mt Weather. Regions, NPSCs, HQ, and other fixed FEMA sites will support
the secondary SMS servers. Microsoft has recommended that the primary to
secondary server relationship be as flat as possible to better control
the software distribution and inventory processes. Therefore, all
secondary systems will be directly subordinate to the primary SMS server
at Mt. Weather. All NEMIS clients will be loaded with SMS client
software to facilitate software distribution, inventory, and remote
network control.

For software distribution, the Tier Three, Level 2 Configuration
Management support team will receive updates and patch fixes from the
Tier Three, Level 3 Configuration Manager and use the primary SMS server
to create a distribution package and a distribution job. The package is
the software upgrade/patch combined with the client instructions for its
installation. These installation instructions can be automated to force
the update and provide all user side response for the installation. The
job is the set of instructions to the SMS system that tell it how to
perform the distribution to the secondary systems. Once the package and
the job are built, the job is executed and the package is passed to the
secondary SMS server. As each SMS client logs into the secondary SMS
server, the package is distributed in accordance with the package
install instructions.

SMS also provides a client inventory component. In the NEMIS
architecture, all SMS clients will periodically report its specific
hardware and software configuration to the primary SMS server at Mt
Weather. This data will be stored in the SMS SQL Server data repository.
The SMS administrative utility on the primary server will be used to
generate enterprise wide client reports against the inventory.

The Tier Three, Level 1 helpdesk and application support team will use
the SMS administrative tool to remote control user desktops. This will
allow the support technician to take over the end-user's PC and show
him/her to use the application.

[^1]: National Emergency Management Information System

[^2]: Tivoli Management Region

[^3]: A managed node is a computer resource under the administrative
    purview of the TMR server. A Managed node can only belong to 1 TMR
    but can be administered from any connected TMR provided the proper
    privileges have been granted.

[^4]: Profile Manager - A group of profiles that are subscribed to as
    one unit by individual profile endpoints. These endpoints can be
    managed nodes or other profile managers

[^5]: Task Library - A feature of TME 10 that enables administrators to
    create and store tasks and jobs that can be run on one or more
    managed resources within the TME environment. It can serve as a
    storage site for binaries, scripts or programs used by TME-based
    applications

[^6]: Profile - A collection of application-specific information.