*DOE/NASA*

*Review Committee Report*

*for the*

Technical, Cost, Schedule, and

Management Review

of the

# Gamma-Ray Large Area Space Telescope

**LARGE AREA TELESCOPE (LAT) PROJECT**

February 2001

EXECUTIVE SUMMARY

On February 13-15, 2001, a Department of Energy (DOE) review committee
conducted a preliminary review of the Large Area Telescope (LAT)
project, the principal scientific instrument on the National Aeronautics
and Space Administration (NASA) Gamma-Ray Large Area Space Telescope
(GLAST) mission. The review was conducted at the Stanford Linear
Accelerator Center (SLAC), the host laboratory for the project. By the
request of the DOE/NASA Joint Oversight Group (Joint Oversight Group),
the review committee was chaired by Daniel R. Lehman, Director of the
Construction Management Support Division, Office of Science.

The charge to the Committee was to: 1) review the technical progress,
including the status of R&D relative to the scientific requirements for
the instrument; 2) examine the proposed budget, cost (and contingency),
and schedule profile, including the commitment of funds and personnel,
to determine its adequacy to complete the project on schedule and on
budget, and 3) evaluate the management structure, including the
relationships to the GLAST mission organization and relationships with
foreign collaborating institutes, as to their adequacy to deliver the
LAT within specifications and budget and on time for the projected
launch date. The following is a description of the LAT systems and the
Committee's findings for each system, relative to the charge.

The objectives of the LAT include the study of the mechanisms of
particle acceleration in active galactic nuclei, pulsars, and supernova
remnants, and the resolution of unidentified sources and diffuse
emissions in the gamma-ray sky. The LAT will also study the high energy
behavior of gamma-ray bursts and transients, as well as probe dark
matter and the early universe. The GLAST LAT will be comprised of four
key components: the tracker, calorimeter, data acquisition, and
anticoincidence detector.

The design for the GLAST tracker consists of a four-by-four array of
tower modules, each with interleaved planes of silicon-strip detectors
and lead converter sheets. Silicon-strip detectors are able to more
precisely track the electron or positron produced from the initial
gamma-ray than other types of detectors. The Committee found that the
concept of the LAT tracker is well matched to the science goals and
utilizes mature technologies. A feasible design was presented that can
be implemented within the available time. A strong consortium of groups
has been established to build the tracker including Italy, Japan, SLAC,
and the University of California at Santa Cruz.

The Calorimeter for GLAST uses Thallium doped Cesium Iodide crystals
primarily because of the abundance of light output. Cesium-iodide bars,
arranged in a segmented manner, give both longitudinal and transverse
information about the energy deposition pattern. Currently, planning is
for only one vendor to be selected to provide the 1,600 crystals
required for the calorimeter. This planning is based primarily on cost
considerations. The calorimeter managers are both from the Naval
Research Laboratory. Other participants include SLAC, as well as
collaborators from France and Sweden. A number of organizational
problems have developed within the French collaboration. As a result,
GLAST funding has been put on hold by the French funding agency. The
French collaboration has been reorganized and is currently in the
process of reapplying for the necessary funding.

The Data Acquisition System records the gamma-ray signals and relays
them to the ground.  Significant components of the Science Analysis
Software have been under development for some time. In particular, the
simulation and reconstruction of particle interactions in the instrument
have been successfully prototyped. This provides a good understanding of
the effort needed in this critical part of the software project. There
is a very good plan but there are several issues that must be clearly
resolved before the Data Acquisition System can be ready for baselining.
Key issues include: 1) definition of project scope and responsibility
especially with respect to calibration and science software; and 2)
pinning down all off-project sources of effort and funding with
Memoranda of Agreement and including the effort in the project.

The Anticoincidence Detector vetoes out the charged particle background
from cosmic ray primary and Earth albedo secondary electrons and
nuclei.  It consists of segmented plastic scintillator tiles, read out
by wave-shifting fibers and photo-multiplier tubes. This technology
matches the performance requirements. Design of the anticoincidence
detector is not yet optimized. Choice of fiber layout in the tiles may
affect the design of the mechanical support. Design of electronics is
relatively more advanced than the tiles.

For integration and testing, the balloon flight program is well along,
and uses completed pieces of tracker, calorimeter, and anticoincidence
detector hardware from previous subtests. The readout software is new,
and the balloon flight provides an opportunity for operational software
checkout and data analysis software development. The flight is scheduled
for June 2001.

The GLAST LAT Project Office (IPO) presented a LAT cost estimate of
\$80.7 million actual year dollars, with an overall contingency of \$23
million, which represents 28.5 percent of the baseline cost. The total
project cost of \$103.7 million is based upon the November 1999
proposal. For the Baseline Review planned for later in 2001, a new
bottoms-up cost and schedule estimate will be prepared. The IPO is
presently implementing a Project Management Control System (PMCS) that
reports cost and schedule performance using an earned value system. The
LAT PMCS is modeled after the SLAC B-Factory project cost and schedule
system, and complies with DOE and NASA management requirements.

The IPO is in the process of finalizing a Memoranda of Agreement with
collaborating institutions and funding agencies, and bringing on more
staff to fill positions in the project. The incomplete status of a
number of Memoranda of Agreement and Memoranda of Understanding has
negatively impacted the schedule. This is an issue for the IPO and the
funding agencies to resolve. The IPO staff is composed of experienced
individuals, with a good mix of expertise and knowledge of large
projects. Some uncertainty exists in tracking the PMCS in the IPO due to
a planned transition from outside consultants to a more permanent team.

As a result of the review, several action items were assigned. By April
1, 2001, the Implementing Agreement between DOE and NASA should be
approved. Also by April 1, DOE, NASA, and the IPO should issue a draft
management plan for comment. By April 15, 2001, IPO is to inform
DOE/NASA as to when they expect to be ready for a baseline review. Based
on information available at this review, it appears that late summer or
early fall would be the timeframe for the baseline review. A deadline of
October 2001 was established.

####### Intentionally Blank

## CONTENTS

Executive Summary i

1\. Introduction 1

1.1 Introduction 1

1.2 Charge to the DOE/NASA Review Committee 2

1.3 Membership of Committee 2

1.4 The Assessment Process 2

2\. Technical Systems Evaluations 5

2.1 Tracker (WBS 4.1.4) 5

> 2.2 Anticoincidence Detector (WBS 4.1.6) 7
>
> 2.3 Calorimeter (WBS 4.1.5) 9
>
> 2.4 Reconstruction and Analysis (Ground Software) 13
>
> 2.5 Trigger and Data Acquisition 17
>
> 2.6 Integration and Test 19

3\. Cost Estimate 23

4\. Schedule and Funding 27

5\. Project Management 29

[Appendices]{.underline}

A. Charge Memorandum

B. Review Participants

C. Review Agenda

D.  Schedule Chart

### Action Items

**1. INTRODUCTION**

**1.1 Background**

The Large Area Telescope (LAT) will be the principal scientific
instrument on the National Aeronautics and Space Administration (NASA)
Gamma-Ray Large Area Space Telescope (GLAST) mission that is scheduled
to be launched in 2005. The LAT is a joint project organized by
Department of Energy (DOE) and NASA supported scientists and
institutions, involving teams from France, Italy, Japan, and Sweden. The
LAT proposal was submitted to and accepted by NASA in response to the
NASA Announcement of Opportunity (AO) 99-OSS-03. The DOE-supported
Stanford Linear Accelerator Center (SLAC) is the host laboratory for the
project and Professor Peter Michelson of Stanford University is the
Instrument Principal Investigator under a NASA contract.

The science objectives of the LAT are largely motivated by discoveries
made by the EGRET experiment aboard the Compton Gamma Ray Satellite and,
for energies above 300 GeV, by ground-based atmospheric Cerenkov
telescopes. These objectives include:

-   studying the mechanisms of particle acceleration in active galactic
    nuclei, pulsars, and supernova remnants;

-   resolving unidentified sources and diffuse emission in the gamma-ray
    sky;

-   determining the high-energy behavior of gamma-ray bursts and
    transients;

-   probing dark matter and the early universe.

The LAT is a gamma-ray telescope based on conversion of the gamma rays
to electron-positron pairs in a stack of tungsten sheets interspersed
with silicon pixel arrays to record the tracks of the particles. The
silicon-tungsten stacks are followed by a segmented cesium iodide
calorimeter to record the energies of the particles. The detector draws
on the strengths of the high-energy physics community, typically
supported by DOE, for the silicon and calorimeter technology and related
physics analysis. Space qualification and telemetry are new dimensions
for high energy physics, but well understood in astro-particle physics,
typically supported by NASA, as well as the foreign collaborators.

NASA and DOE have formed a Joint Oversight Group at the Headquarters
level to coordinate agency oversight of the project. Coordination and
management of the project, including resource management and cost and
schedule accountability and reporting, are supplied by the host
laboratory under the leadership of the Instrument Principal
Investigator. DOE/NASA relationships for the GLAST mission and the LAT
project are formalized in a draft Implementing Arrangement and a draft
Memorandum of Understanding, respectively.

**1.2 Charge to the DOE/NASA Review Committee**

In a December 19, 2000, memorandum (Appendix A) the DOE/NASA Joint
Oversight Group requested Mr. Daniel Lehman, Director, DOE Construction
Management Support Division, to conduct a pre-baseline review of the
GLAST LAT project on February 13-15, 2001. The charge to the Review
Committee was to:

-   Review the technical progress, including the status of R&D, relative
    to the scientific requirements for the instrument.

-   Examine the proposed budget, cost, and schedule profile, including
    the commitment of funds and personnel and the level of contingency,
    to determine its adequacy to complete the project on schedule and on
    budget.

-   Evaluate the management structures, including the relationships to
    the GLAST mission organization, and relationships with foreign
    collaborating institutes, as to their adequacy to deliver the LAT
    within specifications and budget, and on time for the projected
    launch date.

**1.3 Membership of the Committee**

The Committee was chaired by Mr. Daniel R. Lehman, Director of the DOE
Division of Construction Management Support in the Office of Science. It
was organized into eight subcommittees with members drawn from
universities, DOE National laboratories, and NASA Space Flight Centers.
Committee membership and subcommittee structure are shown in Appendix B.

**1.4 The Assessment Process**

The review was the first DOE/NASA review of the LAT project of a
combined series that fulfills the otherwise-separate requirements of the
DOE and NASA management oversight processes.

The review took place on February 13-15, 2001, at SLAC. The first day
was largely devoted in plenary session to overview presentations by LAT
project management and leaders of the detector subprojects. These
presentations were based largely on the Flight Proposal, *GLAST Large
Area Telescope Flight Investigation*, submitted in response to AO
99-OSS-03.

On the second day the members of each subcommittee met in the morning
with their project counterparts to discuss the technical status and
details of the scope, cost, schedule, and management of each subsystem.
The presentations by the subsystems were well prepared and the
discussions were very useful. The afternoon was devoted to subcommittee
discussions and drafting of subcommittee review reports. The closeout
with GLAST LAT management took place on the morning of the third day.

The primary method for assessing technical requirements, cost estimates,
schedules, and adequacy of management structures was comparison with
past experience. Relative to high energy physics detectors with which
the DOE reviewers were familiar, the LAT is a small and simple detector.
The additional complications due to space qualification were familiar to
the NASA reviewers. The cost and schedule basis was, necessarily, the
budget as presented in the Flight Proposal, pending a new, "bottoms-up"
estimate.

####### Intentionally Blank 

**2. TECHNICAL SYSTEMS EVALUATIONS**

**2.1 Tracker (WBS 4.1.4)**

**2.1.1 Findings**

A feasible design for the tracker was presented and can be implemented
within the available time.

The silicon sensors have been designed. A suitable vendor has been
identified and a first fabrication run of prototype devices is underway.

The front-end integrated circuit is relatively simple and can be
implemented with mature processes. The design of the readout integrated
circuit is nearly complete and will be submitted for fabrication within
the next months.

A strong consortium of groups in Italy has taken responsibility for
assembling and testing the ladders, trays, and towers. Plans for
verification and testing are still rudimentary.

A detailed and comprehensive schedule has been developed with the right
elements, but in its current form it is very tight. A bottoms-up cost
estimate by the subproject yields a total cost of \$8.7 million of which
\$4.5 million is labor. The overall estimate appears reasonable. The
total contingency is \$1.7 million and is assigned entirely to materials
and services. No contingency has been included for labor.

A Memorandum of Agreement (MOA) with Japan and University of California,
Santa Cruz has been signed. A draft MOA with the Italian groups exists.

**2.1.2 Comments**

The concept of the LAT Tracker is well matched to the science goals. It
utilizes a mature technology and builds on substantial experience and
development effort in high energy physics.

The tracker design is well advanced. The Committee was presented a
feasible design that can be implemented within the available time. The
required performance is well within the capabilities of standard
technologies.

Specifications for the silicon sensors have been developed by a group
with many years of experience in this field. The Committee is confident
in the design and the selected vendor. A first fabrication run of
prototype devices is underway and devices will be available for testing
soon.

The front end integrated circuit is relatively simple and can be
implemented with mature processes. The design of the readout integrated
circuit is nearly complete and will be submitted for fabrication within
the next months.

A web-based tracking program for monitoring status and location of all
components is under development.

A practical scheme for assembling towers into the grid has been
considered in some detail. Greater modularity of the sensor biasing
scheme would help to avoid loss of a full tower in the event of a noisy
detector or power supply failure. Other opportunities for increased
fault tolerance may exist and further analyses of subsystem-specific
failure modes should be pursued.

Plans for verification and testing are still rudimentary. Much more
thought needs to go into the detailed testing procedures for integrated
circuits, ladders, trays, and towers at the earliest possible time, as
this determines design schedules for test systems. This applies to
electronic, mechanical, and thermal testing, but the Committee sensed
that additional effort is required in electronics. For example, near the
end of assembly phase many tests are proceeding simultaneously and it
must be ensured that sufficient test facilities are available when they
are needed.

Engineering resources for electronics appear to be inadequate,
especially for documentation.

A detailed and comprehensive schedule has been developed with the right
elements. However, its current form is very tight, with delivery of the
final tower scheduled for January 7, 2004, which is seven days beyond
the milestone appearing in the project schedule. Sensor delivery drives
the tower assembly schedule, so increasing the production rate and
introducing more parallelism in the assembly sequence should alleviate
this problem. Potential production bottlenecks must be identified and
followed by mitigation plans. As it stands, the schedule is difficult to
use for monitoring technical progress. Additional "working level"
milestones should be defined.

A preliminary cost estimate was shown. Although insufficient time was
available for a thorough evaluation, the overall estimate appears
reasonable based on the main cost drivers. It was difficult to determine
major cost drivers and allocation of material and labor costs. Overviews
of costs and contingencies broken down into labor and materials and
services at appropriate levels would assist project tracking. Additional
effort for project, cost, and schedule tracking may be necessary.

The MOAs include clear lists of responsibilities and deliverables.

**2.1.3 Recommendations**

> 1\. Identify, at least four month's, explicit schedule contingency for
> delivery of the towers for final assembly.
>
> 2\. Add labor contingency to the cost estimate.
>
> 3\. Develop and review procedures for handling all high-value and
> mission-critical items.
>
> 4\. Develop detailed test procedures beginning now.
>
> 5\. Ensure adequate documentation, particularly for electronics.
>
> 6\. Complete Memorandum of Agreement with the Italian groups.

**2.2 Anticoincidence Detector (WBS 4.1.6)**

## 2.2.1 Findings

***Design Requirements***

The goal of the Anticoincidence Detector (ACD) is to reject charge
particles with an efficiency of 0.9997 (inefficiency of 3\*10-4). This
requires both high light yield tiles and high hermeticity of the
detector (minimal cracks).

Segmentation of ACD should be able to maintain efficiency for gamma
events (reduce self-veto due to backsplash).

Electronic noise has to be low enough not to cause more than one percent
false vetos.

### ***Major milestones for ACD subsystem***  {#major-milestones-for-acd-subsystem .unnumbered}

# ACD & Thermal Blanket Preliminary Design Review June 2001

> ACD & Thermal Blanket Critical Design Review June 2002
>
> ACD Flight Subsystem Assembly Complete October 2003
>
> Flight ACD Ready for Integration at SLAC January 2004

# *Cost* 

At the present time, cost estimate is based on the November 1999 GLAST
proposal. The total cost of the ACD is approximately \$9.5 million (to
be funded by NASA). A contingency analysis has not been performed. Shown
below is the breakdown for the lower level of the

\$9.5 million labor costs:

##  [(on + off project)]{.underline}

## Management \$1.5M 10+8 FTEs

> Quality assurance \$0.5M 1+1 FTEs
>
> Detector construction \$1.4M 2+4 FTEs

## Electronics \$3.8M 9+8 FTEs

# Mechanical Components \$0.8M 6+1 FTEs

# Other (software, I&T, mission) \$1.7M

## 2.2.2 Comments

Technology chosen for the ACD (scintillating tiles with WLS fiber
readout) matches the performance requirements (fast response,
hermeticity, efficiency in cosmic ray rejection, low self veto for
gammas) well.

Design of ACD is not optimized yet. Technical drawings for mounting of
tiles do not exist yet. Choice of fiber layout in the tiles may affect
the design of mechanical support. Design of electronics is relatively
more advanced.

Short-term schedule for ACD is very aggressive. In particular, the
Preliminary Design Review is scheduled for June 2001. Long-term schedule
(ACD delivery to SLAC by January 2004) is comfortable, assuming the
design will be finalized soon.

The cost of the project is uncertain and contingency unknown. In
particular labor estimates for mechanics and quality assurance seem to
be relatively low compared to the management and electronics costs. A
bottoms-up cost estimate is in progress.

**2.2.3 Recommendations**

1.  Perform optimization of the optics design:

> a\. Consider using clear fibers to transmit light between tiles and
> photomultiplier tubes to increase light yield.
>
> b\. Consider a simpler layout of WLS fiber (present design has up to
> 30 fibers/tile, a total 145 tiles) to reduce complexity of fiber
> routing.
>
> 2\. Perform tests (bench measurements/Monte Carlo/Balloon) to prove
> that scintillating fiber ribbons reduce inefficiency of the ACD in the
> crack region (2-3 millimeters) to the level that satisfies the physics
> requirements of the GLAST mission.
>
> 3\. Provide complete technical drawings for mounting of tiles and
> readout photomultiplier tubes on the support structure.

4.  Perform a full bottoms-up cost estimate, including realistic
    fabrication, assembly, and installation scenario, with contingency
    covering possible increases of cost (if design not finalized).

**2.3 Calorimeter (WBS 4.1.5)**

The LAT calorimeter along with the silicon tracker is used to
reconstruct the energy of incident photons. The calorimeter is also
critical to the identification and rejection of background. The LAT will
study gamma rays in the energy range from 10 MeV to 300 GeV. Thallium
(Tl) doped Cesium Iodide (CsI) crystals have been selected as an
appropriate technology, primarily because of the abundance of light
output. The crystals will be read out by two PIN diodes at each end. The
calorimeter will consist of 16 identical modules. Each module will be
comprised of eight layers of twelve crystals deployed in a hodoscopic
arrangement. The calorimeter will be 8.5 radiation lengths deep, so the
energy resolution is dominated by leakage rather than by cracks and
material between crystals. The longitudinal and transverse segmentation
of the calorimeter allows for a determination of the shower shape and a
leakage correction. The longitudinal position in individual crystals can
be determined using light asymmetry measurements from the two ends of
the crystal.

**2.3.1 Findings**

Two PIN diodes are attached to each end of each crystal. This is
complicated by the extremes of temperature and vibration to which the
calorimeter will be subjected. The preferred solution is to glue the
diodes to the crystal face. The optical quality of the PIN diode glue
joints have been shown to degrade during temperature cycling.

It appears that only one vendor will be selected to provide the 1,600
crystals required for the calorimeter. This decision is based primarily
on cost considerations.

The calorimeter has an assigned mass contingency of 2 percent. To stay
within this limit it will be necessary to closely monitor the dimensions
of the crystals. It is generally the case that a significant fraction of
the delivered crystals will have dimensions close to the upper end of
the allotted tolerance. This is due to the production process where the
crystals are cut to the high side of the mechanical tolerance to allow
additional material to be removed during the final polish.

The estimated cost of the U.S. contribution to the calorimeter is
\$11,382.4 K. Significant foreign contributions are also expected. The
contingency for the U.S. contribution appears to be approximately ten
percent. The majority of the costs are for labor rather than materials.

The cost estimate for the calorimeter system has not been updated since
the proposal. Since that time, responsibility for the design of two
Application Specific Integrated Circuits (ASICs) has been shifted from
France to the U.S. (SLAC). It is not clear who is doing the final
procurement and qualification of the production ASICs. This has not been
included in the cost estimate.

The schedule for the calorimeter has been driven by milestones imposed
from the top down and does not appear to be resource loaded. The
schedule is very aggressive in all areas with no contingency. The
schedule does not appear to be consistent with the funding profile. It
is unclear whether the financial resources are currently available to
make sufficient technical progress to meet the Preliminary Design Report
schedule. The schedule for the Preliminary Design Report is very tight.

The analog front end ASIC design was delayed for two years because of
issues internal to the French collaborators. Responsibility for the
design has been transferred to SLAC.

The Calorimeter Subsystem Manager is N. Johnson and the Calorimeter
Project Manager is P. Carosso, both from the Naval Research Laboratory.
The other participating U.S. institution is SLAC. Collaborators from
France and Sweden also play important roles in the calorimeter project.

A number of serious problems have developed between the various French
institutions involved in the calorimeter project. The details were not
explored, but the fallout from these problems has had a significant
negative impact. In particular, GLAST funding has been put on hold by
the French funding agency. The French institutions are currently in the
process of reapplying for the necessary funding to successfully execute
their significant project responsibilities. The front-end analog ASIC
was also a casualty of this situation and sat dormant for two years
before being picked up by SLAC.

2.  **Comments**

The technical knowledge of the calorimeter team is outstanding. They
clearly understand the interplay between the physics goals of the
experiment and the performance requirements of the calorimeter. Many
innovative solutions to technical problems have been developed. The
mechanical structure and electronics interfaces have been particularly
well planned. There is existing expertise to solve the remaining
technical problems.

It is not unusual for crystal orders of this size to be split between
multiple vendors to mitigate schedule delays and technical problems. The
likely vendor has a track record that suggests they can handle an order
of this size but some risk is always present with a single crystal
vendor.

The crystal qualification tests have been well planned. The Swedish
partners are well organized to do the job.

Ten percent contingency is clearly too small. The project is largely
labor driven. Given the tight schedule there should be contingency
available to add additional labor in areas where it might advance the
schedule.

A resource loaded schedule is clearly needed. If the imposed milestones
must be met in order to be ready for a particular launch date, then it
is essential to understand critical paths and areas where additional
resources could advance the schedule.

The French institutions are committed to providing approximately \$17
million to the calorimeter project. They are responsible for procurement
and testing of PIN diodes, attaching the PIN diodes to the crystals and
assembling crystal modules. Continued failure of the French partners to
solidify their organization and commitment places the calorimeter system
at considerable risk. The recent addition of some key personnel to the
French organization does give the committee reason to be optimistic.

2.  **Recommendations**

```{=html}
<!-- -->
```
1.  Sign and implement Memorandum of Agreements with all international
    partners as soon as possible.

2.  Organize the French effort and commitment to agreed upon roles,
    responsibilities, and schedule as soon as possible.

3.  Develop an integrated bottoms-up resource loaded schedule with
    adequate contingency that takes into account the anticipated funding
    profile. Re-evaluate the schedule for the Preliminary Design Report.

4.  Update the cost estimate and assign an adequate contingency.

5.  Resolve the PIN diode glue problems. This should be given a high
    priority by both the Naval Research Laboratory and the French
    institutions.

6.  Define responsibility for procurement, qualification, and testing of
    production ASICs and include in the schedule. The source of funds
    and manpower for this activity should be identified and included in
    the cost estimate.

**2.4 Reconstruction and Analysis (Ground Software)**

## 2.4.1 Findings

Significant components of the Science Analysis Software have been under
development for some time. In particular, the simulation and
reconstruction of particle interactions in the instrument have been
successfully prototyped. This provides a good understanding of the
effort needed in this critical part of the software project.

Because of the rather small data rate, simplicity of the detector, and
low occupancy, the software and computing problem to be solved here is
modest compared to that in large high energy physics experiments.
Nevertheless, the Scientific Analysis Software problem is significant
and critical to the success of the project.

The WBS that was presented was similar to the one in the material the
Committee received initially but the manpower estimate, and therefore
the budget, was reworked and was quite different. The budget will
increase significantly.

The schedule for the Preliminary Design Report for the software was not
consistent with the schedule for other Preliminary Design Reports and,
in fact, seemed to occur after the baseline review.

The major components of the project are:

a.  The Simulation and Reconstruction Software

b.  Analysis Infrastructure and Framework

c.  The Data Production Facility

d.  Scientific and Calibration Software

The first three components are well defined and much progress has
already been made on items "a" and "b." The third component is well
understood but less urgent.

The last component is not as well defined. It is also not clear how much
of the work will be done by this team, how much will be funded by this
project's budget but sourced to other groups within the collaboration or
to foreign groups, and how much will be supplied through the Science
Support Center (a separate project). Similarly, it was not clear what
part of the calibration software and algorithms was to be provided by
this project and what part of that was "off budget."

Resource estimates do not include "off budget" contributions even where
they are important to the success of the project.

The project has chosen to adopt several high energy physics codes,
including Gaudi, Root, and GEANT4, to reduce the total amount of effort.
This is good. However, the Committee feels that there are inadequate
resources for support of these acquired high energy physics products.
These are evolving and in GEANT4's case need to even be debugged at some
level. This effort should not be forgotten or underestimated. Two FTE's,
steady state, is reasonable.

Specific findings with respect to individual WBS items are:

a.  Simulation and reconstruction will be largely complete in next year
    or so. The estimates are credible: two programmers will need to be
    involved in this effort continuously until startup to make
    refinements, support development activities, and to act as back up
    for one another.

b\. Infrastructure and framework. This is always a sink of effort. It
includes such essential but unglamorous activities such as code release,
release management, tools like bug trackers, and database (ORACLE,
MySQL) support. This activity must be supported during the construction
phase

> and will require one or two FTE/programmer(s) at the minimum.
>
> c\. Data Production Facility (software).
>
> This system must be a strong, check-pointed system, with a solid
> database and good documentation so it can be handed off to a lower
> level programmer or operator.

Production for simulation and calibration during the construction must
also be supported. This includes support for the balloon flight. The
Committee found the estimate of three FTE programmers to be quite
credible.

8.  

> d\. Calibration. The Committee believes this is ill defined and the
> resource estimates are too low in the current plan. The estimate for
>
> FTE's includes three scientists doing algorithms and one programmer to
> support them until 2003. After 2003, this reduces to one scientist
> doing algorithm development and 30 percent of an FTE programmer for
> support.
>
> These estimates assume that the detector subsystems will contribute in
> this area.
>
> e\. Science Software. EGRET software provides a model and a good basis
> for making estimates but is too limited to adopt directly. There is a
> detailed "wish list" of software to be written. It requires nine
>
> FTEs, steady state, including the above mentioned calibration
> activity.
>
> Funding of programmers at Goddard through this project has been
> discussed. There is also the possibility of physicists doing this off
> budget
>
> . It is expected that some of the work will be done by the Science
> Support Center. Whatever division is made, an agreement has to be made
> to assure that GLAST gets the software it needs on schedule. The SSC
> and mission physicist goals and schedules may not completely align
> with GLAST.

**2.4.2 Comments**

There is a very good plan but there are several issues that must be
clearly resolved before it can be ready for baselining. Key issues
include:

-   Definition of project scope and responsibility especially with
    respect to calibration and science software.

-   Pinning down all off-project sources of effort and funding with MOAs
    and including the effort in the project.

The group carrying this work forward is very small but is currently
holding its own. To continue to do this, it will need to maintain its
current base and begin to grow quickly. The need to support software
efforts for the design and construction effort makes this even more
urgent. Funding should be secured for the current members of the team,
some of whose support is uncertain at this time.

Some parts of the software covered are very important to the mission and
these are well-defined. These should be assigned a reasonable
contingency in the cost estimate (adding extra effort, etc.) for:

-   Simulation and Reconstruction

-   Analysis Infrastructure and Framework

-   Data Analysis Facility

These activities currently constitute about 40 percent of the effort.
Fifty percent contingency would allow them to add a significant but
manageable number of programmers to deal with schedule slippage in these
key areas.

Other parts of the software are less critical and are also typically
more open-ended. These have "scope contingency" and should be defined as
well as possible but then be viewed as "level of effort" and assigned
only a very small dollar contingency.

On this basis, the Committee estimates a 20 percent overall contingency
for this project if, after review, it continues to be defined with the
full scope that was presented. However, this is simply an estimate based
on the current understanding of the project. Management should work out
the proper contingency once the project is completely defined.

The issue of how much effort will be available "off budget" from
scientists supported through base programs should be understood. In high
energy physics, significant effort is provided in this mode. This is
less true in astrophysics due to the shorter average length of
post-doctoral appointments and to traditions within the field.

"Off budget" effort that is important for completion of the project
should be included in the cost and schedule because it will be
impossible to judge whether the project can succeed without
understanding this. Mechanisms for assuring that this effort is
delivered also must be established.

An integrated total effort of order 75-100 FTE years from all sources
will be required to do this project. This makes it about a \$10 million
effort (much larger than the budget which we were shown which was
claimed to include NASA funding only). In addition, there should be an
appropriate contingency, which is discussed above. The actual budget
could be lower but this depends on a clearer understanding of scope of
the science and calibration software that is to be provided by this
group and by an understanding of how much effort is provided "off
budget" by scientists or foreign collaborators.

The Committee believes that this project is currently on track but must
resolve these issues that have been raised and move to acquire the
necessary staff quickly. Current staff is doing a good job and efforts
should be made to secure their funding.

As a minor comment, the WBS that was presented at this review included
completed projects and projects that will be completed before the
baseline review. This makes the WBS appear more complicated than it
really is. These should be eliminated before the actual review.

## 2.4.3 Recommendations

1.  Develop a resource loaded Cost and Schedule, in time for the
    baseline review, for all aspects of the Science Analysis Software
    including effort that is "off project." This is a required
    resolution for all issues related to project scope.

2.  Develop clear, formal agreements with all off-project software
    providers.

3.  Plan for a sufficient level of infrastructure staffing to track
    changes and development in all the software tools planned for use.

4.  Define parts of software that are mission critical and determine a
    reasonable contingency for those parts.

5.  Plan for calibration software development in conjunction with the
    detector subsystems.

**2.5 Trigger and Data Acquisition**

**2.5.1 Findings**

In preparation for a Preliminary Design Review (expected in August
2001), requirement documents are being prepared for LAT Electronic
Systems, individual subsystems, and some of the components of the
subsystems, notably ASICs.

The technology and design choices that have been made are prudent and do
not involve any speculative technologies.

Power, space, and channel count requirements on the electronics
necessitates the design of several different analog and digital ASICs.
Design of the digital ASICs appears to be well thought out, although the
designs are not yet final, consistent with the maturity of the project.
These ASICs should be straightforward to design by the group of
designers assembled for this project.

-   Three mixed analog and digital ASICs are needed for front end
    electronics of the silicon tracker, calorimeter, and the ACD. The
    responsibility for the calorimeter ASIC has recently been moved to
    SLAC, and a new team is now designing it.

-   The design of the tracker and calorimeter ASIC are very far along,
    with a competent design group identified for each. Each of them has
    had successful partial prototypes. The design of these ASICs appears
    to be very well integrated into the overall system design of the
    LAT.

-   The design of the ACD ASIC is in the preliminary conceptual stage,
    consistent with the progress on the ACD itself. This design should
    progress in the next year to the prototype stage.

Flight software is being designed under the aegis of the designers of
the electronics. A small group has been assembled to design and begin to
implement software to be flown on the instrument, based mainly at SLAC.

The management of this system appears to use the management tools set up
by the project to track budgets and schedule. A very complete schedule
that tracks progress against needs of the detectors has been developed.

**2.5.2 Comments**

A very impressive and competent group of electronic designers has been
assembled to work on this project. The overall system design is
carefully thought out and documented.

The group at SLAC developing electronics for LAT has limited experience
with instrumentation for spaceflight. It may be useful to make regular
consultation with experts in this are available to the electronics
management. Satisfaction of requirements by NASA for documentation,
integration, and verification may require hiring or consulting with
experienced spaceflight professionals.

It is important to manage and track the parts of the electronics being
produced at remote institutions as carefully and thoroughly as the work
at SLAC.

A commendable effort at analysis of failure modes and reliability has
been made, and should continue.

The management of the electronic subsystems has clearly been excellent,
but there are many tasks that will compete for attention in the next
year (preparation of the requirements documents, the design review, the
ballon test, and many ASIC submissions), so any request for additional
resources must be dealt with quickly to keep the project on schedule.

It is important to fill the open position for a software engineer
stationed at SLAC to develop flight software as soon as possible.

**2.5.3 Recommendations**

1.  Carefully consider the need for additional manpower to work on
    flight software based on the estimated needs of this project.

2.  Project management should quickly heed and act upon any requests by
    the management of the electronic subsystem for additional
    assistance.

**2.6 Integration and Test**

This subcommittee reviewed the Mechanical System Design, Integration and
Test, and Balloon Flight portions of the project. The Committee did not
review reliability and quality assurance plans during this review.

**2.6.1 Findings**

The balloon flight program is well along, and uses completed pieces of
tracker, calorimeter, and ACDhardware from previous subtests. The
readout software is new, and the balloon flight provides an opportunity
for operational software checkout and data analysis software
development. The flight is scheduled for June 2001. The effort is nearly
two-thirds complete (by cost), out of an estimate of about \$1 million.

Mechanical system design and integration and test were reviewed
together. There is a common WBS manager for both.

The Mechanical System Design (MSD) task includes: 1) mechanical design
integration, management, spacecraft interface development, reliability,
quality assurance, and mechanical and thermal systems development; 2)
system-level hardware design, including thermal control, radiators, and
the grid; and 3) LAT integration and test support, subsystem integration
and test, verification, LAT, and mission integration and test.

Integration and test task includes: 1) management, coordination and
development of LAT integration, and test activities; 2) reliability and
quality assurance, subsystem verification results; 3) integration and
test preparation, preparation and development of LAT facilities, and
calibration equipment; 4) calibration unit and flight LAT preparation
for integration and test; and 5) mission integration and test support.

## 2.6.2 Comments

Technical resources currently on board for the Balloon Flight effort are
adequate, though there is competition for resources with those needed
for the preliminary design review preparation. Rapid analysis of the
balloon data is essential for maximum value to the project. The program
provides an initial focus for the software efforts.

The Mechanical System Design and Integration and Test efforts have been
centralized in the project office, such that these functions flow down
to the subprojects from the project office. There are lower level shadow
groups in each subproject, and these two groups will coordinate the
overall verification and test plans. Due to this relationship, portions
of the verification tests will be conducted by the same personnel who
designed and built the device. The independence of the testing efforts
is consistent with high energy physics projects, but is less than is
typically applied in NASA efforts.

The project has developed a "plan-for-a-plan" for calibration and
verification activities. Details of this plan have not been completed,
and a working group has been proposed to assemble the components of such
a plan in the coming months. Communication with the subgroups needs to
be established on the same timescale, and capabilities of the subgroups
with respect to verification activities desired confirmed.

This WBS item assumes additional integration and test support at the
final assembly and checkout phase comes from the subprojects. Without a
detailed verification and test plan it will be difficult for this and
the other WBS managers to properly estimate the resources required for
each of the integration and test tasks across the project.

There was considerable discussion of the plan for a final beam test of a
few towers, rather than the whole device. The project personnel
discussed the trade-offs involved, and demonstrated it to be adequate to
the needs of the project. Such a review of each major system test across
the project should be completed, focusing on the technical need and
expectations, and allowing a reasonable time for the analysis of
results.

The thermal and mechanical design of the grid has been through several
iterations and appears adequate at this time in the project.

There is an understanding of the flow of performance and interface
documents between subprojects, the LAT project, and the GLAST project.
Drafts of several high level documents exist. Interface documents do not
exist as yet. A schedule for the production of these documents does not
exist.

Document and information control and dissemination systems are being
established.

There is little apparent schedule float, although there may be hidden
schedule contingency. The philosophy for statistical vs. complete
testing for the components with large numbers has not been formulated,
and directly impacts the test schedules.

It is unclear from the WBS, which boxes in integration and test are to
be staffed and which are purely functional. The flow of money from
several sources and lack of detailed scope definition make it difficult
to evaluate the resources devoted to a single WBS item. The budgets as
presented were \$5 million (for WBS 4.1.8) and \$4.7 million (for WBS
4.1.9).

The subsystem manager presented his personnel estimate, included in the
baseline, including the ramping up of staff in FY 2001 and FY 2002
across these WBS items. The project is actively interviewing, and has
offers on the table to fill two of the currently open positions.

At the current staffing levels, the workload over the next six months
will be heavy and if problems arise, personnel will be stretched thin.
The Committee supports the hiring of additional personnel to join the
capable staff already on hand.

## 2.6.3 Recommendations

1.  Hire an integration and test expert, now scheduled for late FY 2001,
    as soon as possible.

2.  Complete the performance specifications, at least to Rev 0, by the
    time of the Preliminary Design Baseline Review.

3.  Complete the Interface Control Documents, at least to Rev 0, by the
    time of the Preliminary Design Baseline Review.

4.  Create the verification and test plan, working with the other
    subprojects, to confirm the requirements, timing, and resources
    required to implement this plan.

5.  Include testing of the LAT by independent personnel in the
    verification and test plan.

#######  

**3. COST ESTIMATE**

## 3.1 Findings

GLAST management presented a LAT baseline cost estimate of \$80.7
million (real year dollars), with an overall contingency of \$23 million
(real year dollars), which represents 28.5 percent of the baseline cost.
The total project cost of \$103.7 million (real year dollars) is based
upon the November 1999 proposal.

LAT receives funding from two sources, in addition to receiving hardware
from foreign collaborators. While LAT does have some flexibility to
shift spending from one source to another, there are some limitations in
spending funds on certain tasks originating from only one funding
source.

In FY 2000, the project has paid \$4,251 K in actual costs. For FY 2001,
GLAST plans to spend \$11,888 K. This totals to \$16,139 K, or about 20
percent of the planned project cost. Very little contingency cost was
available for FY 2000 and FY 2001.

GLAST management is presently implementing a Project Management Control
System (PMCS) that reports cost and schedule performance using an earned
value system. Cost is generally reported down to the fifth level. The
GLAST PMCS is modeled after the B-Factory cost and schedule system, and
complies with DOE and NASA management requirements.

GLAST management is presently preparing a complete bottoms-up, resource
loaded integrated cost and schedule for the LAT effort. This development
effort is expected to be completed by April 30, 2001. After scrubbing by
GLAST management, this integrated cost and schedule plan will constitute
the baseline for the LAT.

The PMCS team has chosen Primavera P-3 as the schedule database tool,
with COBRA selected for handling the actual costs for the GLAST project
and providing products for external output for NASA and DOE reporting.

## 3.2 Comments

The PMCS team is a very competent team and the Committee thanks them for
their thorough presentation and frank discussion of the present status
and the challenges that they see ahead. It was noted that the PMCS team
developing the baseline is primarily made up of a consulting team (at
least six people from Applied Integration Management) that will leave
the project in April or May 2001.

Resource loading in PMCS is assessed in dollars, regardless of whether
an Materials and Services item or labor effort. This will not allow the
PMCS system to roll-up manpower needs in terms of FTEs, particularly for
DOE scientific support which is supported from base programs.
Contingency, while assessed by the Level 2 subsystems during the
baseline development using a risk and weighted matrix, will be
maintained at Level 1.

### **Table 3-1. LAT DOE & NASA Interim Cost Estimate (Escalated K\$)** {#table-3-1.-lat-doe-nasa-interim-cost-estimate-escalated-k .unnumbered}

**Based upon November 1999 Proposal**

  ----------------- ---------------------------- ---------------- ---------------- -----------------
                                                 **Cost Estmate                    
                                                 (Real-Year                        
                                                 K\$)**                            

  **WBS#**          **Subsystem**                **DOE**          **NASA**         **Total**

  4.1.1             Instrument Management        \$4,633.9        \$1,444.4        \$6,078.3

  4.1.2             System Engineering           \$3,989.6                         \$3,989.6

  4.1.3             (reserved)                                                     

  4.1.4             Tracker                      \$10,965.2                        \$10,965.2

  4.1.5             Calorimeter                                   \$11,382.4       \$11,382.4

  4.1.6             Anti-Coincidence Detector                     \$9,536.1        \$9,536.1

  4.1.7             Electronics & Data           \$2,366.6        \$17,000.4       \$19,367.0
                    Acquisition                                                    

  4.1.8             Mechanical Systems           \$2,177.1        \$2,825.8        \$5,002.9

  4.1.9             Instrument Integration &     \$2,168.0        \$2,491.2        \$4,659.2
                    Test                                                           

  4.1.A             Performance & Safety         \$2,626.0                         \$2,626.0
                    Assurance                                                      

  4.1.B             Instrument Operations Center                  \$2,102.0        \$2,102.0

  4.1.C             Education & Public Outreach                   \$1,503.0        \$1,503.0

  4.1.D             Science Analysis Software                     \$2,718.4        \$2,718.4

  4.1.E             Suborbital Flight (Balloon)                   \$784.9          \$784.9
                    Test                                                           

  **Agency Totals**                              **\$28,926.4**   **\$51,788.6**   

  **LAT Total                                                                      **\$80,715.0**
  Estimated Cost                                                                   
  (TEC)**                                                                          

  **LAT Total                                                                      **\$103,700.0**
  Project Cost                                                                     
  (TPC)**                                                                          

  **Contingency**                                                                  **\$22,985.0**

  **Contingency                                                                    **28.5%**
  (%)**                                                                            
  ----------------- ---------------------------- ---------------- ---------------- -----------------

## 3.3 Recommendations

1.  The core members of the Project Management Control System team
    should be in place as quickly as possible to allow adequate time to
    transfer technology and information from the consultants to the
    permanent team, and to provide a smooth transition into the
    implementation phase of the project.

2.  Complete a bottoms-up, resource loaded cost and schedule estimate
    for the GLAST project to support the baseline review. Estimate
    qualifications should be provided describing the estimate, such as
    quotes or engineering estimates for Materials and Services costs, or
    fully loaded institutional labor rates for manpower costs.

3.  Perform a bottoms-up contingency analysis based upon the appropriate
    parameters (risk and weight factors, etc.) at the lowest WBS for the
    LAT project to support the baseline review. The resulting
    contingencies should be explicitly identified and easily viewed.

4.  Manpower should be tracked explicitly as FTE's in the Project
    Management Control System.

####### Intentionally Blank 

**4. SCHEDULE and FUNDING**

## 4.1 Findings

The integrated cost and schedule baseline consists of about 4,000
schedule activities, and contains a set of milestones consistent with a
launch date of September 2005. This includes a three-month period of
explicit slack identified in the project.

During the implementation phase of LAT, the master P-3 project file will
reside in the Project Office. Monthly updates in earned value (work
performed) will be provided by the subsystem manager (or designee) to
the PMCS staff on a monthly basis.

## 4.2 Comments

When an integrated cost and schedule baseline is established, the
critical path(s) of each subsystem should be identified.

All activities related to off-line testing prior to spacecraft
integration (such as vibration and/or acoustic testing) should be taken
into consideration to ensure the three-month period of slack is against
availability to integrate to the spacecraft.

The LAT has a tight schedule up to the launch date. GLAST management
should use the overall strategy of advancing its work and procurements
whenever possible to increase the schedule slack.

**Table 4-1. LAT DOE & NASA Funding Estimate (Escalated K\$)**

+-------+--------+-------+--------+-------+-------+-------+--------+
|       | **     | **F   | #####  | **F   | **F   | **F   | **T    |
|       | FY00** | Y01** | # FY02 | Y03** | Y04** | Y05** | otal** |
+-------+--------+-------+--------+-------+-------+-------+--------+
| **    | \$3    | \$5,  | \$8    | \$9,  | \$5,  | \$3,  | \$35   |
| DOE** | ,000.0 | 700.0 | ,200.0 | 000.0 | 900.0 | 200.0 | ,000.0 |
+-------+--------+-------+--------+-------+-------+-------+--------+
| **N   | \$3    | \$3,  | \$15   | \$20, | \$18, | \$6,  | \$68   |
| ASA** | ,594.0 | 891.0 | ,500.0 | 300.0 | 900.0 | 500.0 | ,685.0 |
+-------+--------+-------+--------+-------+-------+-------+--------+
| **To  | \$6    | \$9,  | \$23   | \$29, | \$24, | \$9,  | \$103  |
| tal** | ,594.0 | 591.0 | ,700.0 | 300.0 | 800.0 | 700.0 | ,685.0 |
+-------+--------+-------+--------+-------+-------+-------+--------+

## 4.3 Recommendations

1.  Continue to formalize and complete the integrated schedule, focusing
    on establishing the critical paths for each subsystem.

2.  Include all activities in the integrated schedule, including
    non-costed and foreign activities.

3.  Ensure that sufficient slack exists for the individual subsystem
    schedules.

4.  Introduce a milestone hierarchy into Project Management Control
    System.

5.  Develop and implement the GLAST Project Management Plan to capture
    the contingency thresholds and configuration management to control
    schedule changes.

```{=html}
<!-- -->
```
5.  **PROJECT MANAGEMENT**

# 5.1 Findings

The GLAST LAT Project Office (IPO) comprises the Principle Investigator
(IPI), Project Manager, Technical Manager, Instrument Scientist, system
engineers and subsystem managers. LAT management is in the process of
developing documents and tools for managing the LAT project, including
the Project Management Plan and the PMCS. They are in the midst of
finalizing MOA with collaborating institutions and funding agencies, and
are also bringing on more staff to fill positions in the project. The
IPO is managing the development of a baseline for the GLAST LAT project.

# 5.2 Comments

The IPO staff is of high quality; it is composed of experienced
individuals, with a good mix of expertise and knowledge of large
projects. They are putting in place a reasonable set of tools and
documents for managing the project.

There is a lot of work to do in getting ready to baseline this project;
the IPO seems well aware of this. Among these tasks are:

1.  Development of an integrated, resource loaded schedule that takes
    into account the funding profile and schedule constraints. An
    important aspect of the IPO's effort between now and the Baseline
    Review is to develop a realistic schedule for the project, one that
    provides reasonable schedule and cost contingency. This schedule has
    to provide adequate time for Integration and Test.

2.  Growth of the IPO staff to provide adequate system engineering
    resources, with the appropriate mix of skills and the right level of
    effort.

3.  Completion of a Project Management Plan that meets the needs of both
    DOE and NASA, while maintaining as much project management
    flexibility as possible.

4.  Training the managers at all levels to use the management tools the
    IPO is putting at their disposal.

5.  Developing a plan for Risk Management. Members of the IPO convinced
    the committee that they take this hard-to-quantify issue seriously.

The Committee notes that the incomplete status of a number of MOAs and
Memorandum of Understanding has had and will continue to have a negative
impact on the schedule. This is an issue for the IPO and the funding
agencies to work on.

The Committee also noted the need for improved communication between
project management and subsystem management, especially subsystems
managed outside SLAC. More frequent communication at the collaboration
level would also be useful.

# 5.3 Recommendations

1.  NASA and DOE should complete the agreement that establishes the
    Joint Oversight Group as soon as possible.

2.  Complete and approve a Project Management Plan as soon as possible.

3.  NASA and DOE should agree on guidelines for reporting on and
    reviewing the project that do not burden Project Management with
    duplicate requirements.

4.  Develop an integrated, resource loaded schedule and cost estimate.

5.  Develop better means of communicating within the collaboration and
    with subsystems.