*Department of Energy*

*Review Committee Report*

on the

R&D Review

of the

**SUPERNOVA/ ACCELERATION PROBE (SNAP)**

**EXPERIMENT**

July 2002

**EXECUTIVE SUMMARY**

On July 9-11, 2002, a Department of Energy (DOE) Committee conducted a
review of the research and development (R&D) program of the proposed
SuperNova/Acceleration Probe (SNAP) experiment at Lawrence Berkeley
National Laboratory (LBNL). At the request of

John O'Fallon, Director of the Division of High Energy Physics (HEP) in
the DOE Office of Science, the review was conducted and chaired by
Daniel Lehman, Director of the Construction Management Support Division,
DOE Office of Science.

The Committee consisted of seventeen scientific and engineering experts
in the fields of High Energy Physics, Astrophysics, Astronomy, and space
flight. The focus of the review was on the SNAP R&D program in its
current pre-conceptual design phase and its plans to get to Conceptual
Design. In addition, the SNAP team also presented their responses to the
requests for trade-off studies made by the January 2001 DOE review
committee. The charge to the committee was to conduct an integrated
examination of each subsystem, as well as assessing the technical
progress overall, and the cost, schedule, and management planning of the
R&D program, in the context of the entire proposed experiment.

The scientific objective of SNAP is to understand the nature and origin
of the recently (1998) discovered "Dark Energy" causing the acceleration
of the expansion of the universe. The experiment is designed to
precisely measure the history of the accelerations and decelerations of
the expansion of the universe from the current epoch back to
approximately ten billion years ago.

The finding of Dark Energy is of central importance to High Energy
Physics in its mission to explore the fundamental nature of matter,
energy, space, and time. This mysterious Dark Energy does not fit into
the current model of fundamental matter and energy. Further study of
this dominant energy in the universe will address profound issues at the
heart of both cosmology and High Energy Physics and will be of utmost
importance to the understanding of the physical laws and contents of the
universe.

Using the universe as a laboratory, the SNAP experiment is designed to
discover and measure the properties of several thousand type Ia
supernovae (SNeIa) to a precision and distance back in time unattainable
from the ground. These supernovae can be used as "standard candles" and
provide the most direct method of uncovering the nature of Dark Energy.
The design is for a two-meter wide field telescope with an approximately
half-billion-pixel CCD (charge-coupled device for optical wavelengths)
and HgCdTe (Mercury Cadmium Telluride detectors for near infrared
wavelengths) camera. A spectrograph will provide follow-on measurements
of the discovered supernovae.

The Committee assessed whether the proposed project meets the scientific
objectives for determining the expansion history of the universe and
fully probing the nature of the Dark Energy and if the proposed
instrument is able to carry out these objectives. Based upon their
review, the Committee reported that "the need for a space based large
field of view mission like SNAP to elucidate the nature of Dark Energy
via the study of Type Ia supernovae has been convincingly established"
and that "the proposed instruments and observing strategy is appropriate
and sufficient to carry out the scientific objectives of the mission."

It was recommended that the detailed trade-off studies and simulations
presented at the review be published as soon as practical for the
benefit of the wider community. In addition, the Committee recommended
that the SNAP team study weak lensing and other scientific topics
relevant to cosmological parameters that are potentially accessible by
SNAP in order to evaluate their possible inclusion in the SNAP program.

In their detailed review of the instrument concept, the Committee
assessed the technical aspects and major risks of the R&D plan. The
instrument package being developed for SNAP was regarded as "the most
ambitious detector focal plane ever proposed, for ground or space" and
was felt to be the highest risk item of the experiment. The Committee
was impressed with the quality of the LBNL CCD team, though "the CCDs
have not reached full maturity". The infrared team is just getting
started and the Committee commented that the team should ride on the
back of larger efforts currently underway for other experiments while
developing expertise and an infrared testing facility. The spectrograph
design and team are good, with no showstoppers found. The electronics
development was well planned and adequately staffed. Specific
recommendations regarding particular instrument designs, plans or
studies were made. The Committee felt that the CCD team should be
strengthened and that the infrared detector development effort needs to
be strengthened, if necessary by bringing in collaborators with
experience in infrared detectors. Overall, the Committee recommended
that "the development of detectors should be given highest priority of
the entire SNAP project during the R&D phase."

Minor recommendations regarding data transmissions and the attitude
control system for the spacecraft were made by the Committee. The
telescope design planning was found to be on track with no changes
needed. It was felt that SNAP should take advantage of software work
being done elsewhere and that a complete, prioritized list of software
tasks should be developed.

The cost estimate for the planned R&D program appeared to be appropriate
for a project in this phase and the contingency of 34 percent was
adequate, though not generous. Major risks and uncertainties appear to
be adequately identified. It was noted that the significant
contributions of the LBNL Laboratory Directed Research and Development
(LDRD) funds will not be able to continue, due to the three year limit.
The Committee recommended that the project "pursue the requested funding
through the DOE/HEP program office and other potential funding sources."

The management team and structure was found to be well matched,
adequate, and appropriate for the planned R&D phase. The level of
engineering support was excellent. Management tools for cost and
schedule development were found to be adequate. Roles and
responsibilities, as well as R&D deliverables are well defined. The
Committee felt that the SNAP team should establish a
laboratory-to-laboratory agreements with the French groups as soon as
reasonable. The strong support and oversight provided by the LBNL
management was recognized and strongly encouraged to continue. It was
recommended that SNAP management establish discussions with the funding
agencies and continue building the collaboration to enlist groups with
important interest and experience.

Overall, the Committee recommended that "the R&D effort necessary to
move this project forward should be pursued with all possible speed" and
that SNAP is "ready to go to

CD-0, when deemed appropriate by DOE."

**Intentionally Blank**

CONTENTS

Executive Summary i

1\. Introduction 1

2\. Science Requirements (WBS 1.2, 2.5) 5

3\. Space Systems (WBS 1.3, 2.1, 2.3.5, 4.0) 15

4\. Telescope (WBS 2.2) 19

5\. Instrument (WBS 2.3) 21

6\. Ground Segment/Computing (WBS 3.1, 3.2, 5.0) 29

7\. Cost, Schedule, and Funding (WBS 1.1) 33

8\. Project Management (WBS 1.1) 37

[Appendices]{.underline}

A. Charge Memorandum

B. Review Participants

C. Review Agenda

D.  Cost Tables and Schedule Charts

E.  Funding Tables

# **1. INTRODUCTION**

On July 9-11, 2002, the Department of Energy (DOE) conducted a review of
the research and development (R&D) program of the proposed
SuperNova/Acceleration Probe (SNAP) experiment at Lawrence Berkeley
National Laboratory (LBNL). The Division of High Energy Physics (DHEP)
has programmatic oversight responsibility for the proposed SNAP
experiment. The Director of the DHEP, John O'Fallon, requested that the
Director of the Construction Management Support Division, Daniel Lehman,
conduct and chair the review.

The first direct experimental evidence for an accelerating universe,
driven by an unknown energy that permeates space, was measured recently
(1998) by two scientific groups (the Supernova Cosmology Project and the
High-z Supernova Search) using ground-based measurements of the Hubble
diagram from type Ia supernovae (SNeIa). Members of the Supernova
Cosmology Project, centered at LBNL, have continued working on
second-generation measurements using ground- and space-based telescopes
(e.g., the Hubble Space Telescope). In addition, the LBNL-led team
initiated the collaboration that proposed SNAP as a dedicated third
generation follow-on experiment to further explore the nature of the
Dark Energy.

Along with supernovae measurements, cosmic microwave background and
large-scale structure measurements also indicate the presence of Dark
Energy. The current supernovae results, together with these results
imply that about 65 percent of the content of the universe is composed
of this mysterious Dark Energy. The rest is composed of about 30 percent
Dark Matter, leaving only five percent known matter. However, the cosmic
microwave background and large-scale structure methods do not allow
measurements of the nature of the Dark Energy, including whether it's a
constant or has varied over time.

SNAP is a proposed, space-based experiment designed to discover and
precisely measure thousands of SNeIa. From the data collected, the SNAP
collaboration will be able to precisely measure the equation of state of
the universe, measure the history of its accelerations and decelerations
and to study the nature of dark energy, which is causing the current
acceleration of the expansion of the universe.

Using supernovae measurements, SNAP will be able to measure the equation
of state, *w*, of the universe as well as how it varies over time, *w'*.
This detailed knowledge will distinguish between different theoretical
models for the nature of the Dark Energy. For *w* = -1 and constant, it
implies that Einstein's inclusion of a cosmological constant in General
Relativity is correct and there is new knowledge of the quantum vacuum.
For other values and variations over time, it could imply a new energy
in the universe or a new understanding of gravitation beyond General
Relativity. SNAP will be able to differentiate between these
possibilities, as well as other theoretical models for the Dark Energy,
and lead to deeper understanding of the universe.

The main feature of the proposed apparatus is a two-meter, wide-field
telescope with a half-billion-pixel optical and infrared camera launched
into earth orbit. The SNAP experiment began its pre-conceptual planning
phase in 1999. It has been supported significantly both by R&D funds
from DOE and the Laboratory Directed Research and Development funds from
LBNL.

The SNAP experiment has already undergone a number of focused reviews.
An initial review of SNAP was held by the Scientific Assessment Group
for Experiments in Non-Accelerator Physics (SAGENAP) panel in February
2000. At this review, the scientific goals of the experiment were
"deemed extremely important." A blue-ribbon panel review of the R&D
program was conducted by the Division of High Energy Physics in January
2001. The review panel included sixteen scientific and engineering
experts from the fields of High Energy Physics, Astrophysics, and
Astronomy. The Committee reported that the "SNAP science goals are
excellent and address fundamental questions in particle physics and
cosmology." They concluded that further trade-off studies needed to be
done before the next review and recommended that DOE encourage and
support substantial simulation and trade-off studies in this period. The
joint DOE/National Science Foundation High Energy Physics Advisory Panel
2001 Subpanel released their report in January 2002. They endorsed R&D
funding for SNAP from the high-energy physics program. In addition, they
recommended that the full project, if approved, include significant
National Aeronautics and Space Administration (NASA) participation in
the construction and launch of the instrument. The report by the
National Research Council's Committee on the Physics of the Universe,
released in April 2002, recommended three new non-prioritized
initiatives, one of which is to determine the properties of dark energy.
The Committee recommended that "NASA and DOE work together to construct
a wide-field telescope in space to determine the expansion history of
the universe and full probe the nature of the dark energy". The NASA
Structure and Evolution of the Universe 2003 roadmap committee is now on
record endorsing the science goal of measurement of dark energy (though
it did not call out a specific mission design). The SAGENAP panel
reviewed the SNAP experiment again in March 2002 and the report will be
released in August 2002. Recommendations from panels and previous
reviews along with the progress of the SNAP R&D program have developed
to a stage where an extensive review is necessary and forms the basis
for the current review.

The focus of the current review was on the SNAP R&D in its current
pre-conceptual design phase and its plans for the future conceptual
design phase. In addition, the Committee was asked to comment on the
effectiveness of the SNAP team in moving the project to conceptual
design. They reviewed SNAP in the context of the entire proposed
experiment. The Committee was asked to carry out an integrated
examination of each subsystem; the technical progress overall; and cost,
schedule, and management planning of the R&D program.

The Committee (see Appendix B) included seventeen scientific and
engineering experts working in the fields of High Energy Physics,
Astrophysics, Astronomy, and space flight. In addition to their
scientific and engineering expertise, these Committee members had
specific areas of expertise applicable to the SNAP experiment including
space systems, telescopes, electronics, optical and infrared sensors,
large-scale computing, and project management. Observers were in
attendance from both the DOE and NASA agencies.

The Committee reviewed the detailed presentations made by the
collaboration members on the scientific and technical aspects of the
experiment. In addition, they reviewed the collaboration's responses to
requests made by the January 2001 Review Committee. The Committee
provided recommendations to the SNAP collaboration and to the agencies
during the closeout of the review. Their evaluations in terms of
findings, comments and recommendations are contained in this report.

The report begins with a discussion of the science requirements for the
SNAP experiment. The sections that follow (space systems, telescope,
instrument, ground segment, and computing) are organized according to
major subsystems in the work breakdown structure (WBS). The final
sections cover cost, schedule, funding, and management of the entire
planned R&D phase. Appendices at the end of the report show the charge
to the Committee (Appendix A), the review participants (Appendix B), the
agenda (Appendix C), cost tables and schedule charts (Appendix D), and
funding tables (Appendix E). Recommendations resulting from this review
are included at the end of each of the sections.

######## Intentionally Blank

# **2. SCIENCE REQUIREMENTS (WBS 1.2, 2.5)**

## 

## **Findings**

### **Scientific Motivation**

The scientific motivation for the SNAP mission was presented to the
Committee in some detail. The focus of the mission is to study the
cosmological parameters governing the expansion rate of the universe via
the detailed study of Type Ia Supernovae (SNeIa).

The SNeIa are strikingly similar violent explosion events whose physics
can be analyzed in detail from their intensity and spectrum as they
brighten and fade. Most observed SNeIa have nearly the same peak
luminosity, and the variations that do occur are correlated with other
observables and can be calibrated to approximately five percent. Thus
these SNeIa serve as nearly ideal "standard candles" and the observed
variation-corrected peak brightness (magnitude) is then a measure of the
distance to the supernova.

The wavelengths of the photons from the supernova are stretched or
"redshifted" in exact proportion to the expansion of the universe during
the time that the photon travels to earth. Thus the comparison of the
peak brightness (magnitude) and the redshift of the supernova provide a
measurement of the changing rate of expansion of the universe: the
observed magnitude indicates the distance, and hence the time back, to
the supernova explosion, while the redshift measures the total relative
expansion of the universe since that time.

The SNAP mission is designed to obtain a Hubble diagram (peak magnitude
*vs* redshift plot) for a large and well-studied sample of supernovae,
looking back in time over two-thirds of the age of the universe. With
such a history of the expansion rate, it should be possible to determine
the contributions of the decelerating and accelerating energy densities
that make up the universe: the mass energy density Ω~m,~ the vacuum
energy density Ω~∧~, and/or other yet to be studied "dark energy"
components. Such a detailed study can also measure the properties of
this new vacuum energy or dark energy, usually referred to as the
equation of state.

### **The SNAP Mission**

The SNAP mission utilizes a space-based, wide field visible and near
infrared telescope of moderate aperture (2.0 m) to observe SNeIa up to
cosmological redshifts of 1.7. These supernovae, which are the result of
detonating carbon-oxygen white dwarfs, are excellent "standard candles"
and serve as bright distance markers. Over the nominal four-year mission
life, SNAP will observe over 2,000 supernovae over the redshift range
0-1.7. While the intrinsic brightness of an individual supernova may
vary, the large number of observations and high quality spectra will
provide sufficient data to characterize and renormalize the events by
subclass and unambiguously establish the accelerating expansion of the
universe and define the "dark energy" equation of state.

While current day supernova searches have sufficient sensitivity to
observe the effects of acceleration, they are ultimately limited by the
difficulty of ground-based infrared observations, continuous monitoring
of the supernova fields and knowledge of the host environments for each
of the supernova. Utilizing diffraction-limited space optics and a high
Earth orbit that permits passive cooling of the focal plane, the SNAP
mission has Hubble-class sensitivity in both imaging and spectroscopy,
with a camera field of view over 200 times larger than Hubble. With a
four-day cadence, the SNAP scans a 1.3 x 8 degree swath of the celestial
sphere in nine overlapping filter bands from 350-1700 nm. These filters
and the repeated observations permit the creation of continuous light
curves in the important "rest-wavelength" bands established for
supernova study and classification. Periodically, the scanning cadence
is broken to obtain visible and near infrared (NIR) spectroscopy of each
supernova when it is within a few days of peak luminosity (as estimated
from the discovery lightcurves). The spectra will be used to classify
each supernova within the various SNeIa subclasses and to provide
estimates of such quantities such as metalicity, internal host
reddening, and explosion energetics.

The success of the SNAP mission depends on superb control of systematic
effects, both astrophysical and instrumental. The SNAP collaboration has
established a target "floor" of 0.02 magnitudes (two percent) for the
total residual of uncompensated systematic effects. This floor is
several times lower than can be realistically met with alternative
facilities but is essential to an adequate measure of the dark energy
equation of state. Much of the SNAP mission design is devoted to
addressing and compensating for potential systematic effects: the
observing strategy, the layout of the focal plane filters, the
establishment of photometric and spectroscopy markers, and the choice of
target fields. The SNAP science team uses analytic techniques and
simulators to verify that these steps are sufficient.

### **Simulations and Studies**

In the development of major space-based and ground-based science
missions, simulations and special science studies play a critical role
in validating the mission concept and establishing requirements. The
SNAP science team is developing two end-to-end simulations of the SNAP
mission. Both start with a Monte Carlo selection of SNeIa events and
their observable properties. The "light" from these supernovae pass
through the host galaxy (extinction), the intergalactic medium and
surrounding mass concentrations ("grey dust" and weak gravitational
amplification), through the Milky Way (extinction) and reach the SNAP
telescope. In SNAPFast, the first of the two simulations, the telescope
and instrument package are represented analytically, with exposure time
calculators based upon input mission parameters (optical quality,
throughput, etc.) The result is a simulated lightcurve in each filter
and a set of spectroscopic markers. The SNAP science team is developing
a more detailed simulation of the observatory that will emulate the
pixel-to-pixel responses of the camera and spectrograph. After
simulating the science data for the entire four-year mission, the SNAP
team passes the data through simulated operations and data analysis
steps and ultimately produces a composite supernova Hubble diagram
(magnitude *vs* redshift) and obtains values and estimated uncertainties
of the dark energy equation of state using maximum likelihood
techniques. By varying instrumental or astrophysical parameters and
repeating the entire process, the SNAP science team can establish the
importance of each parameter and modify the mission design accordingly.

The SNAP science team has also developed simulations of ground-based and
space-based observatories such as the Large-aperture Synoptic Survey
Telescope (LSST), Hubble, and the Next Generation Space Telescope
(NGST). These simulations include such effects as seeing, atmospheric
emission and absorption, weather, instrumental capabilities, and other
observing constraints. By this technique, the science team can study
whether individual SNAP capabilities can be provided by other
facilities.

### **Response to the January 2001 DOE Review**

The SNAP team presented in detail their responses to the issues and
concerns raised and the recommendations made in the January 2001 DOE
review report (see section 2.2.4).

### **Other Uses of SNAP**

While the current experimental design is optimal for the SNeIa project,
the wide field,

9 filter-band imager is a nearly ideal instrument for other important
scientific studies, such as a weak-lensing survey. A weak-lensing survey
could map the structure in the dark matter and measure its power
spectrum. In addition, other important surveys such as a very deep and
wide photometric redshift galaxy survey with morphology could be done.

## **Comments**

2.  **The Importance of the Scientific Goals of the Mission**

The evidence for cosmic acceleration is shaking the very foundations of
fundamental physics. It focuses the spotlight squarely on the
"cosmological constant problem" which has probably been the most
challenging and awkward problem in theoretical physics ever since the
development of field theory in the mid 20^th^ century. The Supernova
Cosmology Project (SCP) centered at LBNL was one of two collaborations
(SCP and the High-Z Supernova Search) that recently published
measurements of the apparent magnitude versus redshift relation of Type
Ia supernovae. These measurements, together with other measurements of
the cosmic microwave background (CMB) and galaxy clustering, provide
strong evidence that the expansion rate of the universe is accelerating,
rather than slowing down as might be expected for a matter dominated
universe with attractive gravity. To explain an accelerating expansion
rate one has to postulate, in addition to the yet to be observed "dark
matter," a presently unobserved energy density, "dark energy" that fills
space and has unorthodox properties like an effective repulsive gravity
and negative pressure. (The pressure P is related to the energy density
ρ by the equation of state

P = *w*ρ, where *w* could be a negative constant less than --1/3, or
could be a function of the redshift *w*(z), parameterized as *w* =
*w~0~* + *w*' z.) The main scientific goal of the SNAP mission is to
collect a statistically significant sample of SNeIa with detailed enough
information on each to carry out a definitive study of the properties of
this mysterious new "dark energy."

Such a study will address profound issues at the heart of both cosmology
and elementary particle physics. In the words of Professor David
Schramm, to whose memory the recent National Research Council (NRC)
Committee on Physics of the Universe (Turner Report) is dedicated:

> "The study of the very large (cosmology) and the study of the very
> small (elementary particle physics) is coming together."

The importance of the SNAP mission to cosmology is clear in that it can
trace the expansion history of the universe. The SNeIa measurements will
uniquely address issues at the very heart of particle physics in a
number of ways. In an accelerating universe *w* has to be less than
--1/3. If *w* = -1, exactly, the acceleration can be said to be due to a
small but non-zero value of the cosmological constant Λ or a homogenous
false vacuum energy. For other values of *w* there is a completely new
form of energy density, dubbed "Dark Energy." A measurement of *w* by
SNAP (and possibly others) will distinguish between these two
possibilities.

If the acceleration is caused by the cosmological constant Λ, then
particle physics is faced with a severe dilemma: naive expectations
regarding the vacuum quantum fluctuations imply a cosmological constant
term in Einstein's equations of general relativity with a value
approximately 10^120^ times larger than the value of Λ implied from the
cosmological measurements. This disagreement has profound implications
for particle physics---some have called it the most significant problem
known in modern physics. The understanding of this problem is at the
heart of any attempts at unified theories that involve a quantum theory
of gravity. There is some feeling that an exactly zero Λ could be
explained by some as yet undiscovered symmetry law in nature. A small
but non-zero value is very difficult to reconcile with any attempts at a
fundamental theory such as string theory. Recent advances in formulating
a quantum theory of gravity (e.g., string theory) have brought the
problem even greater notoriety because these advances, despite achieving
other impressive successes, have failed to improve the understanding of
cosmological constant, an understanding that is essential for any true
fundamental theory of gravity and matter.

If *w* turns out to be negative but not --1, then particle physics is
faced with a completely new form of energy density, the Dark Energy,
with negative pressure and therefore large-scale repulsion. There is
nothing in the Particle Data Book that would do this. Neither can Dark
Energy likely be explained by other fashionable hypothesized, but in
this context garden variety, forms of matter, like the Higgs boson,
Supersymmetric or Technicolor particles, or the contemplated candidates
for cold or hot Dark Matter. Thus, the existence of this new Dark Energy
would have immense implications for particle physics.

The measurement of the value of *w* would clearly be of fundamental
importance in starting to understanding this new form of energy. Models
have already been proposed (dynamical scalar fields, including
"quintessence," tracker fields, etc.) that can be distinguished by the
redshift evolution of the equation of state parameter, *w* *= w~0~ +
w~1~ z*. SNAP promises to have a special capability of measuring *w~0~*
and *w~1~*, given a value of Ω~m~ and Ω~k~. This will clearly be of
fundamental importance for particle physics, especially since there are
no ideas at this time to investigate Dark Energy employing an
accelerator based approach. It is thus very likely that experimental
cosmology will have a centrally important impact on progress in these
fundamental issues in particle physics. In view of this, it is
appropriate for particle physics in DOE and NSF to help initiate and
realize the SNAP project.

In summary, future developments in fundamental physics depend strongly
on the nature of the cosmic acceleration. If the acceleration is caused
by a non-zero cosmological constant the origin of this non-zero value
must be understood, and our theories of fundamental particles and forces
must be formulated (perhaps re-formulated) in a manner consistent with
this non-zero value. If the acceleration is caused by some other form of
dark energy, then the dark energy must be integrated into understanding
other matter and forces in nature. All these topics lie at the center of
DOE high energy physics research.

The SNAP project provides the best opportunity to determine the nature
of the cosmic acceleration, and optimizes both the maturity of the
method and impact on the fundamental questions. This point is stated
broadly in the NRC Turner Report, and was corroborated by detailed
presentations and discussion during this review. Specifically, the SNAP
project will determine the equation of state of the dark energy, as well
as the time dependence of the equation of state. The targeted SNAP data
product will determine these quantities sufficiently to maximize
discriminatory power among the different explanations of the cosmic
acceleration. Consequently, the SNAP project stands to have a great
impact on some of the most exiting problems facing fundamental physics.

1.  **Mission Need**

As part of the SNAP mission definition and in response to the January
2001 DOE review, the SNAP team has studied the ability of other
ground-based and space-based facilities to accomplish the supernova
survey. In general, the team found that ground-based facilities were
unable to observe a sufficient number of high redshift SNeIa to
constrain *w* and *w*' nearly as well as SNAP. Conversely, observatories
such as Hubble Space Telescope and NGST do not have wide field imaging
capabilities and are thus unable to observe and follow a sufficient
number of supernovae to adequately define the supernovae Hubble diagram.
The team found similar results for combinations of ground and
space-based facilities (excluding SNAP). Weather and the limited
continuous observing zones about the celestial poles constrain the
ability of ground-based observatories to provide prompt spectrographic
follow-up on SNAP supernovae. Limited accessibility and instrumental
capabilities argue against Hubble and NGST being used for a similar
function. (The SNAP mission requires 2000 visible and infrared spectra
during the 2.5-year primary mission.)

Based upon the SNAP science team's simulations and our own experience,
the committee believes that SNAP is *uniquely* capable of accomplishing
a supernova survey that can adequately constrain the dark energy
equation of state. Other facilities will provide a preliminary estimate
for *w* if one assumes that *w*' = 0 as a prior. But only SNAP provides
the necessary wavelength coverage, number of targets, and control of
systematics to provide a quantitative result for *w* and *w*'. Other
independent data are critical for meeting these goals. The Committee
agrees with the project that reasonable priors on Ω~m~ (potentially from
a variety of sources) or data from the upcoming PLANCK mission will
provide complementary cosmological constraints that would greatly
enhance the value of the SNAP results.

### **Instruments and Observing Plan**

Since the January 2001 DOE review, SNAP has refined the focal plane
camera and visible-NIR camera. The current camera design places a matrix
of CCD (charge-coupled device for optical wavelengths) and HgCdTe
(Mercury Cadmium Telluride detectors for near infrared wavelengths)
detectors, each with a dedicated filter(s), in the focal plane. This
design necessitated changes to the observing strategy but provides
excellent control of instrumental systematics (each field is observed by
four independent sets of filter-detector combinations during the year).
The spectrograph features a 3" x 3" image slicer, low spectral
resolution (R \~100), and two spectrographic arms for the visible (CCD)
and NIR (HgCdTe). The designs of the camera and the spectrograph are
driven entirely by the SNAP supernova search. It is believed that these
instruments will accomplish the SNAP mission goals and are essential to
the success of the mission.

The SNAP science team is still refining the details of the observing
plan. They have selected target regions of low galactic extinction and
have adopted the four-day cadence for "mowing" the fields. During the
R&D phase, the project will develop a baseline strategy for inserting
spectrographic observations and explore potential impacts on the quality
of the observed supernova light curves. This is also the appropriate
phase to study the impact of small failures (loss of single detectors,
etc.) The Committee feels that the design of the observing strategy is
sufficient for this phase of the project (trade-off studies and the
setting of initial requirements.)

### **Response to the January 2001 DOE Review**

The SNAP collaboration has made adequate responses to the
recommendations of the January 2001 DOE review.

The need for high redshift (z\>1.5) SNeIa was studied and established
(report by E. Linder). Simulations showed the importance of these SNeIa
in distinguishing dark energy models. This need then drives the need for
good IR imaging and spectral coverage and is a main reason the project
must be done from space.

As recommended, and as discussed in the "Findings" section, much work
was done developing a simulation tool, applying it to optimize the
mission, and comparing it with alternatives. The direct comparison of
SNAP to ground based missions was done under varying assumptions. In all
cases, the clear superiority of SNAP's approach was verified. While
other facilities, such as NGST and ground-based telescopes, can be
useful in some cases, the simulations show that they are not good
replacements for any of the SNAP instruments. Ground-based instruments
cannot get a large sample of high quality 0.9\<z\<1.7 SNeIa. The slew
and settling time of NGST makes it inefficient for obtaining many z\<1.7
SneIa. Follow-up by a ground-based telescope is difficult for much of
the year for SNAP's fields.

As recommended, the team considered descoping the SNAP instruments and
significant degradation in both statistical and systematic errors was
found. The suite of instruments proposed is a good match to the
requirement for a large, uniform sample of SNeIa out to z=1.7, with high
quality spectra, well-sampled light curves, and adequate control of
systematics.

As recommended, SNAP further studied the photometric calibration. The
photometric calibration requirements/estimates are now two percent in
optical, three percent in NIR, and there is a four percent relative
float between optical and NIR. The effects of these calibration errors
were included in the simulations of SNAP's science output. The Committee
believes that this precision is possible to obtain. During the next
phase the project will produce a calibration plan to achieve them.

In response to the recommendation to place the NIR imager at higher
priority, the collaboration increased the HgCdTe detector area to equal
that of the CCD area. This change has improved the instrument and is
incorporated in the end-to-end simulations.

### **R&D Readiness** 

The science optimization and simulation studies presented at this review
were of sufficient extent and depth that the Committee feels that SNAP
is ready to start the R&D phase. The funding requested for the science
study team in the R&D budget appears to be appropriate to complete the
required studies.

### **Other Opportunities**

After SNAP has accomplished its main SNeIa survey, it will remain a
unique instrument and could be used for many other very high priority
scientific studies. While the guest observer program discussed by the
collaboration is not an essential part of the current SNAP mission, the
cost of such a program is probably very small compared to its benefit to
the scientific community.

## **Recommendations**

1.  The need for a space-based, large field of view mission like SNAP to
    elucidate the nature of the Dark Energy via the study of Type Ia
    supernovae has been convincingly established.

2.  The proposed instruments and observing strategy are appropriate and
    sufficient to complete the scientific objectives of the mission.

3.  The detailed simulations, optimizations, and comparisons with
    ground-based facilities that were completed by SNAP and on which the
    above two conclusions are based should be published as soon as
    practical so they may be viewed by the wider community.

4.  The R&D effort necessary to move this project forward should be
    pursued with all possible speed.

5.  The studies of weak lensing and other topics relevant to
    cosmological parameters with SNAP should be pursued further during
    the R&D period.

######## Intentionally Blank

**3. SPACE SYSTEMS (WBS 1.3, 2.1, 2.3.5, 4.0)**

# **3.1 Findings**

## **3.1.1 Attitude Control**

The SNAP project team is aware that the pointing requirements are the
biggest spacecraft issue and is doing the appropriate activities during
the R&D phase to address them. There is some lack of clarity in the way
the attitude control system (ACS) pointing requirements are stated in
some of the documents and presentation materials.

The stability/jitter requirement of 0.01 arc seconds, 1 sigma, is a
*control* requirement, not just a specification for isolation of
disturbances by observatory design.

## **Mission Lifetime**

The primary science goals can be achieved in the stated four-year
mission life. A Guest Observer program that would commence after the
four-year primary mission, was discussed, with no conclusive
requirement, other than stating that it would be of value to the program
and broader science community.

**3.1.3 Ka-band Transmission**

While programs may begin to use Ka-band for data transmission in the
near future, ground stations and transmission equipment do not currently
exist. The SNAP project team is aware of this and is expending effort to
investigate options during the R&D phase.

## **3.1.4 Cost, Schedule, and Workforce**

The overall funding profile and allocated workforce for the space
systems effort seems adequate. The schedule to perform the work also is
adequate.

## **3.1.5 R&D Phase Assessment**

The activities scheduled for the R&D phase address the most critical
issues and are appropriate for this stage of the project.

# **3.2 Comments**

## **3.2.1 Attitude Control**

The spacecraft ACS stability/jitter requirement is specified to be 0.01
arcseconds. A stability/jitter requirement is an allowable delta change
in the spacecraft pointing over a specified period of time. The
Committee was told that the specified period of time was the 300 seconds
target exposure time. Since the period of time specified was the entire
observation period, the 0.01 arcseconds specification is, in reality, a
pointing control requirement and not a stability/jitter requirement. Due
to the control loop response capability and the systematic errors within
a body pointed spacecraft this requirement is difficult to meet. As a
SNAP study showed, it can be accomplished with a fine steering mirror
fast control loop within the body controlled ACS. Hence the previous
committee's recommendation to keep a fine steering mirror within the ACS
trade space.

It is not clear that the stability/jitter requirement was correctly
communicated in the Integrated Mission Design Center (IMDC) study, nor
that the correct comparisons were made when comparing the SNAP
requirements to other missions with similar requirements.

The Committee believes SNAP understands the difficulty of the stringent
pointing requirements and are taking the appropriate steps during their
R&D phase, i.e., the ACS and structural/thermal modeling.

## **3.2.2 Mission Lifetime**

Missions at NASA have a stated design life, used for specifying design
and quality requirements. In addition, a longer goal is usually stated,
to take advantage of the fact that missions usually last longer than
their design life. Consumables are added to operate the observatory
beyond its design life, subject to cost constraints.

## **3.2.3 Ka-Band Transmission**

There are other programs considering using Ka-band for data downlink and
may consider sharing costs to develop equipment or ground stations.

# **3.3 Recommendations**

> 1\. Be clear on the pointing requirements for the Attitude Control.
> State the requirements in a clear, consistent, and understandable way.
> The Committee concurs with the plan to verify the numerical values of
> the requirements during the R&D phase.
>
> 2\. Retain the steering mirror option and include it in the ACS
> modeling. Include it as part of the ACS trade-off studies performed
> during the R&D phase, while the pointing requirements are being
> further defined, and the ACS configuration tradeoffs are being
> studied.
>
> 3\. Perform a cost and complexity vs. scientific benefit trade-off
> study on the requirement to perform dithering.
>
> 4\. Consider sizing the spacecraft consumables (e.g., propellant,
> solar array size, battery depth of discharge, and cycle life) to
> accommodate operating the observatory beyond its design life of four
> years, thereby enabling the support of a potential Guest Observer
> program after the primary objectives of the mission have been
> accomplished. The Committee does not recommend increasing the design
> lifetime, as that will incur large cost increases and the primary
> objectives can be accomplished within the four years. However, to take
> advantage of the fact that missions usually last beyond their design
> life, and to account for the possibility of the Guest Observer
> program, it should be considered to include these consumables in the
> designs now, to see the impact on the designs as the structural,
> thermal, and ACS models are being developed.
>
> 5\. Investigate the possibility of sharing costs with other space
> programs for potential required developments for Ka-band data
> transmission.

**Intentionally Blank**

**4. TELESCOPE (WBS 2.2)**

**4.1 Findings**

The optical design is mature and represents a conservative approach. The
Three Mirror Anastigmat (TMA) is a proven design.

The baseline materials are the classical zero-expansion glasses for the
telescope optics and cyanate ester composites that represent the current
demonstrated technology.

There was a concern that the shutter was a single point failure. The
matter was addressed with a conceptual of a dual shutter, with the
capability to latch a shutter from the closed state, and will be
resolved in the preliminary design phase.

The four-year schedule for the optical telescope assembly (OTA)
construction, assembly, and integration will require continuous
monitoring of the contractors and has no slack.

The Fast Steering Mirror (secondary mirror) was addressed in the answers
to the January 2001 DOE review report. It should remain in the trade
space for possible use for fine guiding and dithering.

**4.2 Comments**

The design of the OTA is classical and proven---there are no technology
development issues.

The primary mirror material should be limited to the zero-expansions
materials presented. Silicon Carbide (SiC) is not as mature in this
aperture. With the schedule as it exists, SiC does not deserve
additional consideration.

Among the zero-expansion materials for the primary mirror Corning's
Ultra-Low Expansion (ULE) glass has two advantages over Schott's Zerodur
and these are lower mass and less depth to the mirror. The latter is
important because it allows more room for the optics behind the primary
mirror. Both benefits relate to the mirrors construction. The ULE
mirrors can be fused or frit bonded together allowing for a rear
facesheet (known as a closed back mirror). These processes are typically
not used for Zerodur. As a result for equivalent bending stiffness, ULE
mirrors are lighter and thinner than Zerodur.

An error budget should be prepared for the optical testing of each
component, and the integrated telescope.

Manpower and funding appears reasonable to reach Critical Decision 1,
Approve Preliminary Baseline Range.

**5. INSTRUMENT (WBS 2.3)**

# **5.1 Findings**

The SNAP project team proposes two instruments, an imager, and a
spectrograph. Both instruments will have optical and near infrared
detectors that are sensitive from 0.35 to 1.7 microns. The imager has a
large field of view (0.7 sq. deg) that is filled with 36 CCDs (3.5K x
3.5K, 10.5 micron pixels) and 36 near infrared detectors (2K x 2K, 18
micron pixels) that will image the sky through nine filters, sampling
the B-band of the rest frame supernova for redshifts of z=0.3 to z=1.7.
The spectrograph is an integral field unit with 20 by 20 spatial pixels
of 0.15 arc sec size whose spectra are sampled with a resolution of 100.
The baseline plan has the same type of detectors for the imager and the
spectrograph. The CCDs are planned to be thick p-channel devices that
are produced by a combination of a commercial silicon foundry and LBNL.
The infrared detector baseline is the Hawaii devices being produced by
the Rockwell Science Center. In order to operate the 900 output channels
of the detectors with low power, the SNAP team proposes use of specially
designed ASICs (application specific integrated circuits) for processing
the video signals from the detectors.

**5.2 Comments**

The detector system being developed for SNAP is the most ambitious
detector focal plane ever proposed, for ground or space. In addition to
the size and complexity of the focal plane, the project is also
developing the CCDs for the focal plane. In comparison to the other
aspects of the mission, the detector systems are the highest risk items
of the mission. The detector development plan is going in the correct
direction but the development plan is optimistic and the team has
presented a success-oriented schedule. To bring the detectors up to the
technical readiness level of the other aspects of the mission and to
keep the overall project schedule, the detector development should be
given highest priority within SNAP.

The Committee believes that the best way to put the detector work in
perspective with the rest of this space mission is to say that this
spacecraft is a "detector project with a telescope attached."

The detector development and production is on the critical path for the
mission. After all development is completed, we estimate that the
production and testing of the detectors will take two to three years.
Another year will be needed to integrate and test the complete detector
system. Since the detector procurement and system integration time
period is comparable to that for the telescope (which was presented as
the longest lead time item), the detector procurement should be
initiated as early as possible in the project. Thus, the goal of the R&D
phase should be to take the detectors completely through the development
cycle, an ambitious goal for a period of one and one half to two years.

Given the criticality of the detector development, it is prudent for the
project to develop second source suppliers for both types of detectors,
and our major recommendations focus on this aspect of the R&D phase. The
plans presented by the SNAP team did not include pursuit of the second
source options. The second source for infrared detectors is obvious
since there are two leading suppliers of devices that SNAP is seeking.
The CCD development and production has three possible scenarios as shown
in Figure 5-1.

**Infrared Detector Development & Production**

**Figure 5-1. CCD Development and Production**

The Committee's recommendations are denoted by boxes with dashed lines.
The Committee's primary recommendation is that a second source should be
investigated for each type of detector. For the CCD development, more
attention needs to be given to the relationship with the silicon foundry
(Dalsa) that is commercializing the LBNL process for the frontside
processing of the detector wafers. Note that the primary source that is
being developed for the CCDs is composed of three steps: frontside
processing at Dalsa, wafer thinning by a third party (not shown in the
diagram), and backside processing and packaging by LBNL. The first
backup option is for LBNL to undertake all of the processing by itself,
something that would require a significant enhancement of the production
capability of LBNL. There are CCD designers and manufacturers that could
independently develop and produce an appropriate CCD for the SNAP
mission (e.g., MIT Lincoln Laboratory). During the R&D phase of SNAP, it
may be prudent to have a backup CCD designed and one lot of CCDs
produced and tested so that a backup option is ready at the end of 2004.
(Note that a four-side buttable package does not need to be developed
for the backup option during the R&D phase.) Also, the CCDs produced by
a different process may not have the features of the LBNL process that
are desired for SNAP (very high QE for 900-1000 nm, tight point spread
function, radiation hardness).

The detector teams, infrared and CCD, need more resources than requested
due to the success-oriented nature of the plan, if the schedule is to be
achieved. The infrared expertise within the team is not at the level
required for a mission of this magnitude and attention should be given
to strengthening the infrared detector expertise within the SNAP
project, if necessary by bringing in collaborators with experience in
infrared detectors.

**5.2.1 Charged-Coupled Devices**

The charged-coupled device (CCD) development is primarily under the
control of LBNL. While at first blush this would appear to be
comforting, on closer inspection it raises concerns. There is a large
amount of work to be done to complete the development of the CCDs,
including: 1) new CCD designs and iteration of the designs after
testing, 2) transfer of the fabrication process to a commercial foundry,
3) design and implementation of a number of new features in the devices,
4) debugging of the thinning and backside processing, 5) development and
testing of a new four-side-buttable package, and 6) development of
filter technology to be placed directly on the CCD surface. In addition,
the CCD team needs to develop a new generation CCD testbench that can
measure quantum efficiency and automatically test and optimize the
devices; needed both for development of the CCDs and in preparation for
large scale testing during the procurement. In addition, the CCD team
has many critical roles being filled by a single person. Steve Holland
is the CCD designer, the liaison to the commercial foundry and one of
two persons that works in the LBNL wafer processing facility. SNAP
should consider hiring a CCD expert to work closely with Steve Holland
for information transfer and off-loading of some of the current
workload.

One of the new features that is being attempted with the CCDs is
modification of the design to produce a tighter point spread function
(PSF). The tighter PSF is produced by significantly increasing the bias
voltage on the backside of the device, a design and operation change
that is difficult. While this is an admirable goal, the Committee was
not certain this effort and risk is necessary since a wider PSF affects
the blue band measurements most---the blue band the signal will be the
most intense and will not be adversely affected by a slightly wider PSF.

To the Committee's knowledge, no one has ever placed scientific quality
interference filters directly on science quality CCDs. There is
significant effort and testing required to do this properly, if it can
be done at all, and the staffing plan does not seem to adequately allow
for the effort required in this area.

The CCD team should more proactively engage the ground-based telescope
community in the testing of the SNAP CCDs. The SNAP project team can get
a large number of very experienced and talented people testing SNAP CCDs
at no cost to the project via a distribution of CCDs to observatories
for use in their instruments. Distribution of detectors to ground-based
telescopes was instrumental in the success of the first Hubble Space
Telescope detectors. The distribution of CCDs should proceed as soon as
feasible, but should commence no later than the end of 2002. In
undertaking a CCD distribution, the Committee recommends that a SNAP
acknowledgement be required in all papers published with data from SNAP
CCDs. Prior to the distribution of CCDs, LBNL must address the question
of which organizations are given access---specifically, should devices
be distributed to foreign observatories?

The LBNL CCDs have not reached full maturity and although the Committee
was very impressed by the quality of the LBNL CCD team, it would be
prudent to investigate an entirely different source of CCDs; n-type CCDs
designed, produced, and packaged by a third party manufacturer. A backup
design and fabrication run could be accomplished during the R&D phase.

**5.2.2 Infrared**

The infrared detector development is riding on the back of the much
larger efforts being undertaken for Hubble Space Telescope (WFC3), NGST
and ground-based telescopes (ESO, CFHT, Gemini, etc.). The detector team
should let those groups lead the way but should develop infrared
detector expertise and follow the developments by procuring and
characterizing some devices.

The 1.7 micron cut-off HgCdTe arrays for Hubble Space Telescope /WFC3,
designed to be operable at the relatively warm focal plane temperatures
achievable for WFC3 (and also for SNAP) currently face significant
technical challenges: significant instability (hysteresis) for the high
QE parts that have been fabricated, as well as excessive read noise. The
WFC3 project will push very hard on these performance parameters, but
the SNAP project team should be aware that these issues are not yet
resolved and should be monitored closely. SNAP performance requirements
should be carefully thought through and clearly articulated. There is
concern about the readout noise level of the newest Rockwell detectors,
and the SNAP team should clearly identify what noise level is required.
The SNAP team can have confidence that the noise performance will be
pushed very hard by the project listed above, and there should not be
significant concern about the readout noise at this time. A second
possible source of infrared detectors that should be investigated is
Raytheon Infrared Operations (RIO).

The SNAP infrared detector team is only beginning to set up a laboratory
for testing infrared devices, a process that is harder than would first
appear, especially if the goal is to test the detectors at the highest
level. The SNAP team should ensure that they have within their
collaboration an infrared testing facility equal to those at other
leading organizations (e.g., ESO).

**5.2.3 Spectrograph**

The spectrograph design is efficient, the team from France is very good
and there appear to be no major concerns with the spectrograph
conceptual design. There is the possibility of a custom-designed CCD for
the optical arm of the spectrograph, with the optimal size, shape, and
number of pixels. The spectrographic CCD could incorporate frame
transfer that would provide two important capabilities: 1) shutterless
operation and, 2) very slow readout during subsequent exposure to
achieve lowest noise. With the custom-designed CCD, the spectrograph
could be operated in shutterless mode, either as primary mode of
operation or as backup in case of shutter failure.

**5.2.4 Electronics**

The electronics development is well planned and adequately staffed,
provided the persons listed in the plan are available as scheduled to do
the work. A conceptual design of the front-end ASIC (application
specific integrated circuit) exists and although challenging with
respect to noise performance, is well within the expertise of the
designers involved. An ASIC technology has already been evaluated and
seems to satisfy the SNAP performance requirements regarding radiation,
noise, and operating temperature.

The preliminary plan for the overall electronics system is to store
minimally processed data in a solid-state recorder and transmit the data
to ground once per orbit via a high-bandwidth K-band datalink. The
Committee thinks this concept is good and has many advantages.

The Committee felt that the power budget allowed for the electronics
system seemed low and the SNAP electronics team should investigate
whether the power and thermal system can provide more power if needed.

The design team should include analysis of electromagnetic interference
and shielding requirements during the design phase.

The failure modes and failure rates of the detectors and associated
electronic system need to be investigated. For example, how many
channels or even sections of the system can be lost and still meet the
scientific mission goals? This answer will affect the partitioning and
redundancy of the system electronics.

The design of the power supply system should be included in the
conceptual design phase, since some of the bias supplies, as well as the
low-noise, low-voltage supplies may have to be a full-custom design.

**5.3 Recommendations**

1.  The development of the detectors should be given the highest
    priority of the entire SNAP project during the R&D phase.

2.  Strengthen the infrared detector development effort, if necessary by
    bringing in collaborators with experience in infrared detectors.

3.  Strengthen the CCD team to have a higher probability of meeting the
    schedule.

4.  Give additional attention to the ongoing relationship with the
    silicon foundry that is commercializing the LBNL CCD process.

5.  Investigate an n-type CCD designer / manufacturer as an independent
    backup option.

6.  Revisit the requirement for the presently planned point spread
    function (PSF) for the CCD.

7.  Upgrade the CCD test facility to provide timely feedback during the
    development cycle.

8.  Revisit the plan for depositing filters directly on the detectors.

9.  Distribute the SNAP CCDs to several more ground-based telescopes to
    increase testing of the detectors.

10. Investigate a second source vendor of infrared detectors.

11. Investigate a custom-designed CCD with the optimal format for the
    spectrograph.

12. Study the feasibility of eliminating the shutters from the
    spectrograph.

13. Address the failure modes and failure rates of the detector
    electronic system, including the detectors.

14. Perform as planned a system test with a prototype ASIC (application
    specific integrated circuit) and a CCD to verify that 16-bit dynamic
    range can be obtained.

15. Develop a calibration plan for the electronics system, especially in
    light of the planned (and supported by the committee) use of an
    on-chip multi-range ADC concept.

######### Intentionally Blank

**6. GROUND SEGMENT/COMPUTING (WBS 3.1, 3.2)\
**\
**6.1 Findings and Comments**

**6.1.1 Mission Operations (WBS 3.1)**

Development of the Mission Operations support is based on the foundation
of the Berkeley Ground Station (BGS). The BGS seems to keep itself, its
equipment, and software, up-to-date. The also have a good procedure for
training operators, a great deal of experience, plans for end-to-end
test, etc. BGS will form an excellent base for support of the SNAP
mission.

The number of R&D issues is small and includes: 1) Ka band antenna
(upgrade or new), 2) secondary downlink and method for transferring data
to LBNL, and 3) determination of amount of buffering and required
hardware in case of failure or unavailability of link to SOC. R&D issues
are correctly identified and the resources requested should be able to
accomplish the few R&D goals.

**6.1.2 Science Operations Center (WBS 3.2)**

Scope Definition. While the task of defining the project scope has
begun, much work remains to be done. The goal is to get a credible
scope, cost, and schedule for the data processing and analysis component
of the Conceptual Design Report. The human resources assigned to this
task are about 3.5 full-time equivalents (FTE) per year.

Project Definition. Science Operations Center/Software is now included
as a high-level subproject of the R&D project and will be treated as
such in the full SNAP construction project. A subproject leader is in
place, other key management roles have been identified and filled, and
personnel have been identified to carry out the R&D. A WBS for the R&D
subproject has been developed.

Fast Simulation Effort. The SNAPFast effort has succeeded and has helped
the SNAP project team understand many key requirements and design
issues. It has also helped them understand the capabilities of other
approaches to studying SNeIa.

Detailed Simulation Effort.  The next phase of simulation is a detailed
pixel level simulation of the instrument, which is necessary both to
help make key design choices and to test the scientific software.  The
effort allocated to this task is about 3.5 FTEs.

Hardware is not a major problem. The project could use Parallel
Distributed Systems Facility (PDSF)/National Energy Research Scientific
Computing Center today. The PDSF facility is modernized every three
years. Also, LBNL also now supports "mini farms/facilities" if SNAP
needs a dedicated facility.

Staffing Levels. The project has about 6.5 FTEs per year for the R&D
part of the Science Software. They have names next to all "slots" except
one graduate student to be named later. The average duty factor of
people involved is slightly less then 0.5. But a good part, nearly half,
of these people (and half the effort) will likely be committed to
writing the detailed simulation, so the resources available for this
scope definition are only 3-3.5 FTEs per year.

It is highly appropriate that Science Operations Software be treated as
a major aspect of SNAP and be managed as part of the project. Effective
software will be required both to optimize the final design of SNAP and,
in operation, to realize the full potential of the telescope, especially
through calibration and control and understanding of systematics. 
Failure to acknowledge the complexity and cost of the software effort
has caused serious problems for similar projects.

Depending on how much software needs to be written as opposed to being
obtained from elsewhere, the final project could be fairly small or very
large. If most software had to be developed from scratch, the cost and
size of this project can be estimated by comparison with similar
projects and could well exceed \$20 million and require 25 FTEs for
several years; such projects have proved extremely hard to manage
successfully.

Scope Definition.  Defining the scope of the scientific software project
is a key goal. It is a difficult but necessary job. To get
accurate buy-steal/adapt/build decisions, which the Committee
recommends, will require actually trying various codes to see how much
functionality is missing and how much effort would be required before
they met requirements.

Many of the decisions will depend on who will be available
for programming. Good people are needed, to keep the enterprise centered
on scientific rather than computer science goals. The project manager
described needed staff as: 1) scientists who also do programming, and 2)
programmers cognizant of the scientific aspects of the project. The
Committee concurs with this statement.

Data access is an important aspect of the scope definition. How
will data access be handled for science for the group and for
wider dissemination? This is important for doing the science and is a
place where other projects have had problems.  The SNAP project needs to
decide whether the data distribution plan should be designed to only
handle the main SNeIa dataset or also to support the ancillary science.

The Committee also observed that in evaluating "external" code for
possible use in SNAP, due attention should be paid to maintenance and
support issues and lifetime costs. It is hard to know whether 3.5 FTEs
per year is adequate to the task of defining the scope of the software.

The only way to know if an instrument is working is to push it to
its limits.  The Committee encourages the SNAP software team not to
concentrate too single-mindedly on the photometry of SneIa. If the codes
are designed only to achieve SNAP's core scientific goal, there is
concern that subtle problems in the data or processing that have shown
up in a broader scientific analysis may not be found.

The project leader believes that 3.5 FTEs are sufficient to complete the
detailed simulation program in about one year if everything goes well.
He would feel more comfortable if there were about 50 percent more
effort. If the software is not completed in roughly a year, it may not
be useful for the development of the Conceptual Design Report. It is, in
any case, needed for testing.

In order to be useful in the design phase, the simulation
development must be paralleled by an effort to produce realistic
analysis software. This does not appear to be part of the plan and we
could not tell whether the existing science code is adequate.

**6.2 Recommendations\
     **

1.  Strive to control the size and scope of the software effort by
    taking advantage of work done elsewhere.

2.  Develop a complete, prioritized list of software tasks. The project
    should identify a list of the most promising potential external
    sources of code for each task and make a plan for evaluating these
    packages to see which can be used or adapted for use. The project
    should also attempt to make an estimate of the level of effort
    required to adopt, adapt or build from scratch. A report on this
    activity should be one deliverable for the next review.

**Intentionally Blank**

**7. COST, SCHEDULE, and FUNDING (WBS 1.1)**

# **7.1 Findings**

The SNAP project has made substantial progress in developing its R&D
cost estimate since the January 2001 DOE review. The estimate has been
developed in accordance with a well-defined hierarchical work breakdown
structure (WBS). The various subproject estimates are collected and
organized in a standardized cost control system that facilitates
drill--down to lower levels in the WBS and takes into account
appropriate wage rates, overheads, and escalation rates.

The project currently estimates the costs (in thousands) of the R&D
phase over the next two fiscal years (FY 2003, FY 2004) is shown in
Table 7-1.

**Table 7-1. R&D Phase Estimate**

+------+------------------+-------------+-------------+--------------+
| **W  | **Description**  | ## **FY03** | ### **FY04  | #### Total   |
| BS** |                  |             | ** {#fy04 . |              |
|      |                  |             | unnumbered} |              |
+------+------------------+-------------+-------------+--------------+
| 1.1  | Management       | 599         | 808         | 1,407        |
+------+------------------+-------------+-------------+--------------+
| 1.2  | Science          | 543         | 935         | 1,479        |
+------+------------------+-------------+-------------+--------------+
| 1.3  | Systems Eng      | 871         | 1,479       | 2,350        |
+------+------------------+-------------+-------------+--------------+
| 2.2  | Telescope        | 479         | 908         | 1,387        |
+------+------------------+-------------+-------------+--------------+
| 2.3  | Instrumentation  | 3,445       | 5,419       | 8,864        |
+------+------------------+-------------+-------------+--------------+
| 2.5  | Calibration      | 434         | 581         | 1,015        |
+------+------------------+-------------+-------------+--------------+
| 3.2  | Science Ops      | 535         | 556         | 1,091        |
+------+------------------+-------------+-------------+--------------+
|      | **SubTotal**     | **6,907**   | **10,685**  | **17,592**   |
+------+------------------+-------------+-------------+--------------+
| 1.0  | Contingency      | 593         | 5,315       | 5,908        |
|      | (34%)            |             |             |              |
+------+------------------+-------------+-------------+--------------+
|      | **Total**        | **7,500**   | **16,000**  | **23,500**   |
+------+------------------+-------------+-------------+--------------+

Contingency for the project has been estimated by reviewing the cost
estimates at appropriate levels of the WBS and typically applying one of
two values: 30 percent for engineering estimates or 40 percent for
physicist estimates. Separate (and potentially different) contingencies
on equipment and labor were not assumed. Contingency dollars were
determined after the addition of any general and administrative (G&A)
costs to the base costs. An effective contingency of 34 percent or
\$5,908 K resulted. The project has allocated this contingency at \$593
K in FY 2003 and \$5,315 K in FY 2004.

Within the proposed two years of R&D funding, there is about an
eighteen-month period to develop the Conceptual Design Report, starting
in January 2003 and concluding in August 2004. The first three months of
FY03 assume continued R&D funding at current (FY02) levels. FY04
concludes with two months of R&D funding corresponding to approximately
45 FTEs on-project and eight FTEs base program and \$650 K per month to
maintain the workforce for the transition into the preliminary design
phase.

There are two specific exclusions from the \$23.5 million R&D cost
estimate that contribute significant value to the project but are not
considered part of the R&D costs. These are:

a.  Certain DOE/HEP base program labor and equipment costs.

b.  In-kind contributions to spectrograph development from French
    institutions that are members of the SNAP collaboration.

Preliminary R&D schedules using MicroSoft Project exist or are being
developed for each Level 2 system. These schedules have not been
resource-loaded.

# **7.2 Comments**

The level of detail of the R&D cost estimate is appropriate for a
project in the pre-conceptual planning phase. As the cost estimate is
developed further during later project phases, the project office should
consolidate the backup documentation that contributes to the basis of
estimate (BOE).

The overall contingency, 34 percent, is judged to be adequate, though
not generous, for a project about to embark on a conceptual design phase
that involves significant R&D. The risks and uncertainties have been
adequately identified.

The asymmetric allocation of contingency between FY 2003 and FY 2004
implies that the work in FY 2003 is very well defined and essentially
all the uncertainties are in FY 2004.

The proposed R&D funding profile during the next two fiscal years seems
reasonable and allows for smooth transitions into and out of the
Conceptual Design Review phase. Failure to secure the necessary funding
according to the plan could adversely impact the schedule and/or
increase the risk and, potentially, the final cost of the project.

The lack of resource loading in the subsystem schedules is not
considered to be a problem at the present time, given the current R&D
scope and level of schedule detail. However, as the project progresses
to its subsequent phases, the proper resource-loading of a comprehensive
and more detailed schedule should help in projecting resource needs,
identifying possible over-allocations, and assessing any resulting
schedule impacts.

# **7.3 Recommendation**

1.  The project should pursue the requested R&D funding through the
    DOE/HEP program office and other potential funding sources.

######## Intentionally Blank 

**8. PROJECT MANAGEMENT (WBS 1.1)**

**8.1 Findings and Comments**

H. Heetderks has taken over the Project Manager position and R.
DiGennaro has become Deputy Project Manager.

NASA is not currently part of the SNAP Collaboration.

SNAP support now includes a significant contribution from LBNL funds
that cannot be continued into FY 2003.

SNAP is reaching out to others with relevant experience, such as the
Space Telescope Science Institute, to learn appropriately and possibly
enlist collaborators.

Project Management of the R&D Phase include:

-   The new Project Managers brings extensive relevant experience to
    bear.

-   The level of engineering support seems excellent.

-   The manpower level of the management appears appropriate

-   The management tools for cost and schedule development and control
    seem adequate. Management roles and responsibilities are well
    defined. R&D deliverables have been defined, and if achieved, will
    result in a successful R&D phase. However, any significant shortfall
    to the funding request would jeopardize these goals and either delay
    the project or increase risk.

-   The currently formulated SNAP Project Management is well matched and
    appropriate to the SNAP R&D phase.

The GLAST project at SLAC has generated some relevant experience as a
DOE laboratory led satellite project. Some of their experience may be
useful to SNAP. SNAP has benefited by a tightly focused science program
to keep technical problems and costs at a reasonable level. NASA might
contribute to the success of SNAP with several key technologies and be
valuable collaborators on the science.

**8.2 Recommendations**

1.  Establish a laboratory-to-laboratory Memorandum of Understanding for
    the Spectrograph R&D effort as soon as reasonable. DOE should begin
    planning with CNES for a timely agreement for subsequent work.

2.  The SNAP Science Principal Investigators (PI's), DOE, and NASA
    should establish discussions regarding collaboration on SNAP.

3.  Continue building the collaboration to enlist those with important
    interest and experience in the community.

4.  LBNL management should continue to provide support and oversight to
    SNAP.