*Department of Energy/*

*National Science Foundation*

*Review Committee Report*

*for the*

Technical, Cost, Schedule, and

Management Review

of the

DARK ENERGY

SURVEY (DES) PROJECT

### January 2008 

###  {#section .unnumbered}

## **EXECUTIVE SUMMARY**

A Department of Energy (DOE) Office of Science (SC) and National Science
Foundation (NSF) review of the proposed Dark Energy Survey (DES) project
was conducted on January 29-31, 2008 at Fermi National Accelerator
Laboratory. The review was conducted at the request of Dr. Dennis Kovar,
Associate Director for High Energy Physics, DOE/SC, and Dr. G. Wayne Van
Citters, Director, Division of Astronomical Sciences, NSF, and was
chaired by Mr. Daniel Lehman, Office of Project Assessment, DOE/SC. For
DOE, the purpose of this review was to assess the readiness to establish
the technical, cost, and schedule baseline of the Dark Energy Camera
(DECam) project in preparation for Critical Decision (CD) 2, Approve
Performance Baseline. In addition, the project plans to seek CD-3a,
Approve Long Lead Procurement and a limited set of other fabrication
activities. For NSF, the review served to assess the progress and status
of the proposed Blanco Telescope at the Cerro-Tololo Inter American
Observatory (CTIO) facilities improvement project (CFIP) and data
management system (DESDM) projects.

The Committee recommended approving CD-2/3a for DECam, the DOE
responsibility, pending resolution of the recommendations in their
report.

The DES experiment proposes to utilize the existing CTIO in Chile. The
construction project includes the new camera, DECam, optimized for the
study of dark energy, the CFIP upgrades and a DESDM. The DES is
requesting funding for the DECam (from DOE) and funding for the DESDM
and CFIP (from NSF). Funding for CFIP would be provided through the
National Optical Astronomy Observatory (NOAO), CTIO's parent
organization. In addition, funds are being requested from other U.S. and
foreign institutions.

The DECam technical design was found to be satisfactory to meet the
performance requirements. The procurement and fabrication plans are
appropriate and the contingency is adequate. The Committee was concerned
about the costs of engineering and integration and recommended they be
reviewed to ensure sufficient support and contingency. The project is
requesting CD-3a for long-lead time items: Charge-Coupled Devices (CCD)
processing and packaging starting in April 2008 and hexapod procurement
in October 2008. The Committee recommended a finite element modal
analysis to validate the design of the hexapod prior to procurement.
They also recommended completion of noise tests and a review of the
Aluminum Nitride (AlN) boards for the CCD packaging prior to full
production orders.

The DESDM project has made significant progress since the May 2007
DOE/NSF review though there were still areas that are not well-defined.
The Committee felt that the staffing plan was very lean and recommended
that the Work Breakdown Structure (WBS) and costs be reevaluated. The
CFIP technical designs satisfy the performance requirements.

The DECam cost baseline of \$32.9 million is supported by detailed and
documented cost and contingency estimates. Increases from the CD-1
(Approve Alternative Selection and Cost Range) estimated Total Project
Cost (TPC) of \$25.0 million were due to increased equipment costs
(\$1.5 million), incorporation of university engineering labor (\$1.1
million) and incorporation of the R&D starting at CD-0 (\$5.3 million).
The MIE has a contingency of \$5.1 million (32 percent). The project
requested CD-3a (Long Lead Procurements) at a cost of approximately
\$2.1 million. The DECam project presented a funding-constrained
schedule based upon a bottoms-up resource-loaded schedule. The schedule
contingency for the CD-4 milestone is twelve months, which was deemed to
be adequate.

The DESDM team presented a TPC of \$7.6 million using a mix of funding
sources. The Committee recommended review of their staffing plan, which
may affect costs. The project presented a resource-loaded schedule that
follows a series of spiral development cycles and has a schedule
contingency of nine months. NSF funding for R&D has already been
awarded. The project submitted a \$2.9 million proposal to NSF for three
years starting in FY 2009 and a decision will be made by NSF pending
completion of the recommendations for reevaluating the costs.

The CFIP project is estimated at \$860K. The team presented a summary
schedule with approximately twelve months of schedule contingency, which
was deemed to be adequate. The project is expected to be funded out of
the CTIO operating budget, contingent upon NSF approval.

ES&H aspects are properly addressed with appropriate hazard analyses and
safety documentation completed for this stage of the project.

The project team has most of the tools in place to successfully manage
the project. The global Memorandum of Understanding between the
collaborating institutions is comprehensive and should be signed as soon
as possible.

There were no specific Action Items resulting from the review.

**CONTENTS**

Executive Summary i

1\. Introduction 1

2\. Science 3

3\. Technical 9

3.1 Dark Energy Camera (DECam) 9

3.1.1 Optics, Optical Corrector, Mechanical Systems 9

3.1.2 Charged-Coupled Devices (CCD) Focal Plane and Camera 13

3.1.3 SISPI and Front End Electronics 18

2.  DES Data Management (DESDM) and DECam Simulations 23

3.3 CTIO Facilities Improvement Project (CFIP) and Integration 28

4\. Plan for Community Use of DES Systems, Installation, Commissioning
and Operations 31

5\. Environment, Safety and Health 35

6\. Cost 37

7\. Schedule and Funding 39

8\. Management 41

[Appendices]{.underline}

A. Charge Memorandum

B. Review Participants

C. Review Agenda

D. Cost, Schedule, and Funding Tables

E. Management Chart

F. EIR Lines of Inquiry

#### 1. INTRODUCTION

The discovery that the expansion of the universe is accelerating was
first announced in 1998. Studying the nature of the "dark energy"
causing this has become one of the most important science objectives in
physics and astronomy. This mysterious dark energy comprises about 70
percent of the matter-energy contents of the universe. Not much is known
about it other than that it exists.

Determining the nature of dark energy is a high priority science
objective for the Office of High Energy Physics. The Dark Energy Survey
(DES) experiment offers a cost-effective and timely approach to
unraveling the mystery of dark energy.

The DES experiment will use the existing Blanco Telescope at the
Cerro-Tololo Inter American Observatory (CTIO) in Chile to study the
nature of dark energy. The project includes the fabrication of a new
camera (DECam) optimized for the study of dark energy, the CTIO
facilities improvement project (CFIP) for upgrade to the telescope and
CTIO facility, and a data management system (DESDM). The DES project
plan calls for funding for the DECam to be provided by the Department of
Energy (DOE) and funding for the DESDM and CFIP by the National Science
Foundantion (NSF). Funding for CFIP would be provided through the
National Optical Astronomy Observatory (NOAO), CTIO's parent
organization. In addition, funds are expected to be provided from other
U.S. and foreign institutions.

The DES collaboration consists of scientists from Fermi National
Accelerator Laboratory (Fermilab), Argonne National Laboratory (ANL),
Lawrence Berkeley National Laboratory (LBNL), the NOAO, five U.S.
universities, and institutions in Brazil, Spain, and the U.K.

In exchange for providing the instrumentation, the DES collaboration
will get 30 percent of the Blanco telescope observing time over a
five-year period. DES will provide significant science results using all
four methods recommended by the Dark Energy Task Force: supernovae,
galaxy clusters, baryon acoustic oscillations, and weak lensing. The
main method is the use of galaxy clusters. They plan to combine their
data with other telescope experiments such as the South Pole Telescope
to enhance their results. Their data will improve the figure-of-merit
well above the recommended factor for a Stage III experiment.

For DOE, the purpose of this review was to assess the readiness to
establish the technical, cost, and schedule baseline of the DECam
project in preparation for Critical Decision (CD) 2 approval. In
addition, the project plans to seek CD-3a, Approval Long Lead
Procurement, and a limited set of other fabrication activities. For NSF,
the review served to assess the progress and status of the proposed CFIP
and DESDM projects, and to inform the decision about further funding by
reviewing a pending proposal.

**2. SCIENCE**

**2.1 Findings**

The context of the science case for the DES is presented in the report
"Dark Energy Task Force" (2006, astro-ph0609051) commissioned by DOE,
NSF, and National Aeronautic Space Administration (NASA). The Dark
Energy Task Force (DETF) defined different stages for the studies of
dark energy:

-   Stage I represents what is now known about dark energy;

-   Stage II represents the anticipated state of knowledge upon the
    completion of ongoing projects that are relevant to dark-energy;

-   Stage III comprises near-term, medium cost, currently proposed
    projects; and

-   Stage IV comprises a Large Survey Telescope (LST) and /or the Square
    Kilometer Array (SKA), and/or a Joint Dark Energy (Space) Mission
    (JDEM).

In particular, the DETF recommended:

> *"II. We recommend that the dark energy program have multiple
> techniques at every stage, at least one of which is a probe sensitive
> to the growth of cosmological structure in the form of galaxies and
> clusters of galaxies."*
>
> *"III. We recommend that the dark energy program include a combination
> of techniques from one or more Stage III projects designed to achieve,
> in combination, at least a factor of three gain over Stage II in the
> DETF figure of merit, based on critical appraisals of likely
> statistical and systematic uncertainties."*
>
> *"Smaller, faster programs (Stage III) are needed to provide the
> experience on which the long-term projects can build. These projects
> can reduce systematic uncertainties that could otherwise impede the
> larger projects, and at the same time make important advances in our
> knowledge of dark energy."*
>
> *"13. Six types of Stage-III projects have been considered. They
> include:*

a.  *A BAO survey on a 4-m class telescope using photo-z's.*

b.  *A BAO survey on an 8-m class telescope employing spectroscopy*

c.  *A CL survey on a 4-m class telescope obtaining optical photo-z's
    for clusters detected in ground-based SZ surveys.*

d.  *A SN survey on a 4-m class telescope using photo-z's.*

e.  *A SN survey on a 4-m class telescope employing spectroscopy from an
    8-m class telescope*

f.  *A WL survey on a 4-m class telescope using photo-z's.*

> *These projects are typically projected by proponents to cost in the
> range of tens of millions of dollars. (Cost projections were not
> independently checked by the DETF."*

The DES will survey 5,000 sq-deg in the filters *grizY* and a repeated
deep pointing of approximately ten sq-deg for ten percent of the time to
find and follow approximately 1000 supernovae in *riz* in the redshift
range 0.3\<z\<1. To do these surveys, the DES will build the DECam,
which is a 3 deg-sq camera using fully depleted CCDs. The survey will
use the NOAO Blanco 4m telescope at CTIO, using 30 percent of the
telescope (525 nights) from 2011-2016. The DECam is scheduled to be
delivered to CTIO in December 2010, on the telescope in March 2011 for
the first science images, and final handover by April 2012. The rest of
the time, the DECam will be made available to the NOAO user community
through the usual telescope time allocation process at NOAO.

The DETF report, released in July 2006, recommended that one or more
mid-term experiments with a combination of techniques be done, and that
(in combination) they should achieve at least a factor of three gain in
the DETF figure of merit over current projects.

The DES will measure properties of dark energy using the four probes of
dark energy recommended by the DETF---supernovae, galaxy clusters,
baryon acoustic oscillations, and weak lensing. The 10-sigma limiting
magnitudes along with a two percent photometric error will measure
photometric redshifts for individual galaxies to a precision of 0.1 in
z, and to 0.02 for galaxy clusters. The DES will select clusters and
estimate their mass using weak lensing, Sunyaev-Zel'dovich (SZ)
detection from the South Polar Telescope (SPT), and optical galaxy
cluster identification. The DES will cover the full SPT field. Using an
equation of state formulation of w(z) = w~0~ + w~a~(1-a), the estimated
increase in the figure of merit from the expected Stage II experiments
will be 4.6, consistent with the findings of the DETF for Stage III
experiments.

The DECam can reach to higher redshifts because the fully depleted CCDs
have high quantum efficiency past 1 micron, and produce no fringe
patterns. The DES will collaborate with the European Southern
Observatory Visible and Infrared Survey Telescope (VISTA) and its VISTA
Hemisphere Survey (VHS) that will provide *JHK~s~* photometry of the DES
survey region using the VISTA 4m telescope in Chile. This will improve
the photo-z's measurements from a 1 error of 0.073 to 0.059, assuming 10
limits of (*g,r,i,z*) = (24.6, 24.1, 24.0, 23.9), and greatly improve
the redshift errors beyond z\~1.3. Since the May 2007 DOE/NSF review,
the DES has signed a Memorandum of Understanding (MOU) with the VISTA
project to add *JHK* photometric data from the VHS to the DES database,
in return for access to the DES database.

The photometric redshifts are important for the four tests, and
extensive atlases of spectra are needed to train the photometric
redshift technique. At present, the DES will include the spectroscopic
and photometric catalogs of SDSS equatorial stripe 82, CDF-South, SNLS
D1/Virmos VLT, XMM-LSS, and ELIAS S1, to help calibrate these important
photometric redshifts. The SDSS field is particularly important to
verify the photometric accuracy required in the Science Requirements
Document. It is expected that at the time of the start of the DES there
will be roughly 400,000 spectra in the magnitude range covered by the
DES from future spectroscopic surveys.

**2.2 Comments**

The Committee found that the science case for the DES clearly meets the
definition of a Stage III mission as described in the DETF report. There
are two parts to the Stage III mission, and they have satisfied both: 1)
the Figure of Merit based on the four main experiments of the DES, along
with certain assumed priors, will achieve an estimated gain of 4.6 in
the FoM, well above the factor of three recommended by the DETF, and 2)
the project has done an excellent job of anticipating and modeling the
known systematic errors in the various aspects of the experiment.
Understanding these systematics and controlling them are extremely
important for the Stage IV experiments such as the JDEM and the LSST.

The scientific techniques for studying dark energy via the four major
science routes as laid out by the DES continues to be of primary
importance in the study of dark energy. The DES remains a leader in the
Stage III science projects and no other large experiment has been
proposed to affect this leadership on the time scale for the completion
of the project. It is clear, however, that the instrument must be
pursued on a time line as given to the Committee in order for this
project to make a large impact in Stage III science.

DES was conservative in developing science requirements in order to
match technical constraints, including matching the science to what the
telescope can safely deliver. In other words, there continue to be no
large looming technical requirements that could compromise the science
goals. As stated in the May 2007 DOE/NSF review report, "*the DES
maintains a healthy advantage in getting the science done, provided that
the funding does not slip."*

Since the May 2007 DOE/NSF review, the Committee noted significant
progress in: formation and co-ordination of science working groups;
organizing access routes to the U.S.; a much strengthened supernova
science case; inclusion of weather patterns in the modeling of the
observations; and a mature science requirements document.

At the time of the CD-1 review, the Committee made a number of
recommendations to be considered prior to CD-2 approval. The Committee
found that the project has answered and met these recommendations, and
summarized their responses and concurred with their conclusions:

1.  *Explore a program to improve Blanco telescope image quality from
    0.9" to 0.7" Evaluate such a program in terms of incremental cost to
    the project, incremental science gains, and the possibility of real
    improvement in the image quality.*

> A goal of median seeing of 0.8" was adopted by the DES, while
> maintaining the requirement of 0.9".

2.  *Convene an NSF, NOAO, and DOE common users advisory group to
    recommend the U.S. user's community needs for the use of DECam.*

> The DES/Ferimlab/NCSA/NOAO MOU created a process whereby NOAO uses its
> existing advisory committee to examine the community needs. NOAO will
> canvas the community through the NOAO Users Committee.

3.  *Resolve science overlap between DES science goals and user
    community science goals in the case where the user community
    proposes for time on DECam to do the DES science.*

> The NOAO Time Assignment Committee will receive a yearly report on the
> scientific status of the DES, which will be considered in the TAC
> process. The Committee felt that this is the best way to gauge overlap
> with the DES. The NOAO TAC can then approve projects that are
> synergistic with the DES goals, or even overlap with the goals if the
> science is compelling. This will allow free access to scientific
> inquiry for the U.S. users of NOAO.

4.  *Actively review progress of competitors to maintain timeline
    towards scientific goals.*

> The DES maintains contact with competing projects through conferences
> and informal discussions.

5.  *Determine the uniqueness of the science beyond achieving the factor
    of three goal of Stage III experiments.*

> The uniqueness of the DES was laid out in the DES answer to the May
> 2007 DOE/NSF recommendations, and will not be repeated here.

6.  *Consider making the goal of one percent relative photometry as best
    effort goal with two percent minimum goal.*

> The project adopted a goal of photometric accuracy at the one percent
> level. This is especially important for the supernova science case.

7.  *Review and expand the goals of the supernova science component.
    Consider varying the cadence of data in the wide field survey to
    maximize sensitivity to SN science.*

> The DES has established a new SN working group, and has shown
> significant progress in revising the SN science case to fit the
> overall DES observing strategies.

8.  *Organize a coordinated theory group across all the science
    projects.*

> It is expected that the Simulations Working Group, which is composed
> mainly of theorists from the other working groups, will evolve into
> the theory group.

9.  *Create a plan for the DES if the SPT telescope for some reason does
    not deliver the SZ clusters.*

> The SPT project should be completed well before the end of the DES
> project, allowing for ample time to modify the science goals if there
> is failure with the SPT. If the SPT for some reason does not deliver
> the SZ galaxy clusters, much of the science can still be done with a
> cluster catalog selected from optical+NIR from VHS and DECam
> observations. The wide-area survey, while originally designed around
> SZ clusters, is very well suited for weak lensing and large-scale
> structure (BAO) science.

10. *Produce quantitative goals for the improvement of systematic errors
    for the Stage IV experiments, as outlined in the DETF.*

> These goals are set out in the draft of the new Science Requirements
> document.
>
> Given the science case presented to the Science Committee members and
> the answers to the May 2007 DOE/NSF review recommendations, the
> Committee found that the science case clearly passes the requirements
> for CD-2 approval.
>
> The Committee provided the following recommendations. These
> recommendations are of a minor nature and do not have to be resolved
> prior to CD-2 approval. They should be revisited at the time of CD-3
> approval.

**2.3 Recommendations**

1.  Finalize the science requirements document within two months.

2.  Convene the Supernova Science Working Group to review the basic
    science goals of the supernova case. Special attention should be
    given to utilizing the unique features of the DES. A fully developed
    supernova science case should be made within six months.

3.  Create test observing schedules considering the observing cadences,
    image quality, and photometric weather goals to plan how the time
    will be divided between the four science cases. This should be done
    within six months.

4.  Canvas DES partners for other minor science cases to see if they
    will fit into the DES observing strategies.

5.  Consider allowing the detailed nightly observation logs (date, time,
    filter, exposure time, weather quality, seeing) to be available to
    the U.S. community in real time for the creation of synergistic
    science.

**3. TECHNICAL**

# 3.1 Dark Energy Camera (DECam)

# 

# 3.1.1 Optics, Optical Corrector, Mechanical Systems 

# 

# 3.1.1.1 Findings

Work Breakdown Structure (WBS) tasks in the area of optical and
opto-mechanical assemblies are complete with resource loading and
documented cost estimates.

Six of nine recommendations from the May 2007 DOE/NSF review have been
fully addressed. Pending items include a stray light analysis and the
design of the C1 lens cover. The two tasks will be contracted to outside
engineering groups. Finite element analysis of the combined Dark Energy
Camera (DECam)/telescope structure was performed but did not include a
modal analysis. This will be completed by the project team.

The DECam project hired a Project Scientist (50 percent FTE), whose
principal responsibility is translating DES science requirements into
technical requirements. System engineering documents (requirements and
interface documents) are being developed and configuration control is
being implemented.

The procedures for manufacturing the DECam barrel assembly are adequate.
The process relies on controlling the buildup of tolerances to ensure
that optical components are aligned with sufficient accuracy. The
spacing of optical components will be adjusted during assembly of the
DECam barrel using machined spacers to optimize for variations in the
as-built lenses from nominal. An end-to-end optical test is not planned.

After a review of different types of mechanisms, the project elected to
use a hexapod for controlling DECam position within the cage assembly.
The decision is based on a review of requirements that concluded
five-axis motion is required to maintain the optical alignment of DECam
relative to the telescope primary mirror in the presence of gravity and
thermal deflections of the telescope structure. The project has received
engineering and fabrication cost estimates from a number of vendors.
Hexapod development and procurement is near the critical path. The
project is preparing to issue a request-for-quotes for an engineering
study leading to fabrication and is requesting CD-3a approval to go
forward with this long lead-time assembly.

Glass blanks have been received for the C1-C5 lenses that comprise the
imaging system of DECam. The University College of London has
responsibility for managing the polishing and coating of the lenses and
for mounting the lenses in cells and final installation and alignment of
the optics in the barrel assembly. This critical work is being performed
as an in-kind contribution. The lenses will be polished and coated by a
commercial vendor. Quotes have been obtained from optical fabricators
and UCL is preparing to move forward with the polishing pending final
approval from its funding agency.

The filter changer and shutter assemblies are being provided by
University of Michigan. The assemblies are based on scaling up of
existing designs. The technical and schedule risk for these items is
deemed to be low.

The DECam interference filters are larger than any previous filters of
this type. Filter vendors have thus far been unwilling to commit to the
tight specifications for uniformity established by the project. The
project is still discussing this with the vendors but it is likely the
filter specs will have to be relaxed. While this may complicate the data
reduction, it is not expected to compromise the science but this should
be discussed with the DESDM project.

The procurement of filters is on the critical path due to funding
restrictions. No spares are currently planned.

At the time of CD-1 the top ring and cage assembly were rotated in the
telescope on a trunion to switch between DECam and f8 operation. This
mechanism was eliminated in the current design. The f8 secondary mirror
will be attached to the lower end of the DECam cage assembly and the
mirror will be removed from the telescope when the DECam is in use.
Removable counterweights will be used to maintain the telescope balance.
A changer mechanism ("f8 Manipulator") will be provided to move the
secondary mirror into position for insertion and removal. This change
opens up space behind the camera assembly.

Project thermal specifications allow the DECam structure to vary by up
to 2°C from ambient. A shroud around the barrel assembly traps excess
heat, which is removed by a cooling system.

# 3.1.1.2 Comments

Cost estimates appear to be conservative for the most part. Contingency
allowances on equipment and labor are based on established procedures
that take risk into account.

Regularly spaced milestones plus other metrics are sufficient for
tracking progress. Slack is not explicitly included in the WBS. No
schedule problems were identified at the time of the review; however,
the time and expense allowed for assembly and integration of the DECam
on the telescope simulator at Fermilab and on site at CTIO appear
optimistic and should be thoroughly reviewed prior to CD-2 baselining of
the budget.

System engineering improved within the DECam project with the hiring of
a project scientist. Still, it is not clear how system engineering is
coordinated within the overall DES project. The Committee judged that
the document tree representing the traceability of requirements, both
forward (top-down) and backward (bottom-up), lags the overall technical
development of the project. While the design appears sound, at no time
during the review was an error tree or error budget presented to
substantiate the technical design, hence budget and schedule. The
Committee urged that this step be completed as soon as possible.

A FE modal analysis of DECam assembly mounted in the telescope top ring
is needed to validate the design of the cage, vanes, and hexapod. The
analysis will provide the basis for specifying stiffness requirements
for the hexapod assembly. It may also reveal problems in the cage/vane
design that lower modal performance that should be addressed before
final design. Specifically, the subcommittee for opto-mechanical systems
noted that the load path between the DECam through the hexapod to the
support vanes was not as direct as an optimal path. The analysis should
be completed prior to the hexapod being approved for CD-3a. The results
of this analysis should be incorporated into the overall image quality
error budget. The Committee noted that the current assignment of zero
contribution from wind buffeting ("assuming a calm night") to the image
budget is not realistic given the site wind statistics at CTIO.

A cage temperature of 2°C above ambient is likely to produce significant
degradation in image quality (cf. Zago, 1995). It is difficult to assess
the true impact of this without a comprehensive image error budget and
analysis but the thermal design should be reviewed in light of this
effect.

Time did not permit an adequate discussion of the proposed f8
Manipulator concept. The Manipulator rises roughly 14 feet above the
floor and passes through telescope structure. Personnel engaged in the
mirror removal will ride up and down on platforms attached to the lift.
The Committee had some questions regarding operational aspects and
safety of the proposed concept. Potential hazards include collision with
the telescope and other structure, pinch points, large seismic loads and
failure of the mechanism with people on board. A review of the concept
with a discussion of safety issues in coordination with the CTIO staff
is warranted.

The control path for the hexapod is not well defined. This starts with
the analysis of wavefront measurements taken in the focal plane and the
translation of those measurements into corrective motions of the DECam
cage. Active figure control of the primary mirror is also an option if
the wavefront sensing is sufficiently accurate. Such techniques are used
on the Magellan and LBT telescopes and most new telescopes now in
planning. Well-established numerical methods can be applied to invert
the wavefront error matrix to corrective actions needed to maintain
alignment and figure control. A review of these methods is warranted
given the stringent requirements on image quality and shape imposed by
the weak lensing science case. The exact method of determining the
corrective actions will affect the requirements for wavefront sensing,
thus will influence the design of the F&G CCDs in the camera.

Advancing the procurement schedule for the interference filters by one
year would reduce schedule risk. The lack of spares in the event a
filter is broken will have a direct impact on the science that can be
done until a replacement is obtained. Filters are particularly
susceptible to damage due to their fragility and the frequency with
which they are handled during operations. The DECam project should
evaluate the programmatic impact of the loss of a filter and obtain a
quote for the cost of providing spares. Filters purchased in pairs
should have lower unit costs and better matching characteristics.

# 3.1.1.3 Recommendations

1.  Continue to implement the systems engineering controls and
    documentation prior to CD-3b.

2.  Complete the pending recommendations from the May review prior to
    CD-3b.

3.  Review the costs for DECam assembly and integration at Fermilab and
    CTIO and provide a generous contingency to the budget and schedule
    for these activities prior to CD-2 budget baselining.

4.  Perform an FE modal analysis of DECam in the telescope upper
    structure.  Use the results of the analysis to confirm the stiffness
    of the hexapod/cage/vane assembly and set specifications for the
    required hexapod stiffness.  Incorporate those specifications in the
    request-for-quotes being sent to potential hexapod vendors.

5.  Analyze the effect on image quality of a hot air plume caused by the
    cage assembly at 2°C above ambient prior to CD-3b.

6.  Investigate the cost to provide a set of spare filters prior to
    CD-3b.

7.  Develop a deterministic strategy to control the hexapod positioner
    in the required five degrees of freedom prior to CD-3b.

8.  Proceed to CD-2 with opto-mechanical systems.

9.  Proceed with CD-3a procurement of the hexapod assembly pending the
    successful completion of the hexapod engineering study demonstrating
    the proposed design complies with project specifications.

# 3.1.2 Charged-Coupled Devices (CCD) Focal Plane and Camera 

# 

# 3.1.2.1 Findings and Comments

***CCD Manufacturing***

CCD fabrication is progressing well at DALSA and LBNL, with fabrication
rate being limited by the funding profile.

The packaging of the CCDs at Fermilab is now a mature process with good
results (e.g., CCD height and flatness) and minimal impact on yield.

Overall yield is acceptable (26 percent) and now known to a fidelity,
which is appropriate to this stage of the project; however, it is
commonplace in low volume production to experience major yield loss that
cannot be predicted from the sample size available to make yield
estimates. For example, the fabrication at LBNL is conducted on
(sometimes old) machines for which there are no spares so that a single
process failure could cause either multiple CCD losses or long delays.

***CCD Testing***

Previous recommendations for improving the test hardware and software
were followed. The testing program is in good shape, providing
comprehensive and accurate measurement capability, automated operation,
and adequate safety margin for throughput. The Committee applauded the
use of database tools to store and collate test data, allowing, for
example, histograms of a given performance parameter across all or a
selected set of devices to be plotted. Keep up the good work.

***Focal Plane Assembly***

A well-planned scheme was presented for safe installation of CCDs on
(and attachment to) the focal plane support plate, which protects the
CCDs, will produce a flat focal surface and relieve stresses due to
differential contraction between the Invar CCD package and the Aluminum
support plate. Thermal FEA was presented that predicts a slight bow of
the focal plate (approximately 10m) due to thermal gradients.

***Dewar and Vacuum System***

The internal volume and surface area of DECam is not exceptionally large
for an astronomical instrument. The unusual feature is the large amount
of Kapton, which will outgas water vapor.

An ion pump was selected as the preferred alternative on the basis that
getters present reliability/maintenance issues. The ion pump requires
electronics, which will not work in a power outage and will glow while
operating.

Water vapor, which will account for the pressure in the dewar will
rapidly freeze out at start of cooling and thus does not affect vacuum
performance. It is important to remove this water, nonetheless, since it
can cryopump onto the detectors when the cooling system is turned off if
the detectors warm more slowly than the surfaces onto which the water
has frozen.

The majority of astronomical CCD systems use getters, and do so
successfully. The reports that getters present operational problems are
probably based on examples of operator error.

While zeolite will indeed lose some pumping ability after adsorbing
water vapor its capacity for water adsorption is enormous, and is
generally only seriously degraded when the zeolite is exposed to
atmosphere for many hours. A vacuum hatch should be provided for easy
access to the zeolite container, and the dewar should be backfilled with
dry gas prior to being opened.

The activated carbon getter regenerates at room temperature so that
optimum pumping of air is assured. When used in combination with carbon,
the role of the zeolite is to remove water vapor from the vacuum when
the dewar is warm and valved off and not to remove gases when cold
(though it will generally also do this) so the combination is very
reliable.

Since the zeolite will rid the dewar of water vapor when warm and
valved-off, the need for long pumping times, and/or large diameter
valves and pumping lines is alleviated.

The principal challenge with getters is to provide fine enough mesh on
the container to prevent particles from escaping, however this has been
done successfully.

The four-inch gate valve with a large diameter hose to an on-instrument
turbo pump is a magnificent but is an unnecessarily fast pumping
arrangement. Water vapor can be more effectively removed by bleeding a
small amount of dry air into the vacuum either continuously or
intermittently during initial pump down, and can be further accelerated
by heating the dewar walls. Once the bulk of the water vapor has been
pumped, the dewar can be valved off leaving the zeolite to adsorb any
residual water vapor.

The zeolite will not be saturated in this process, but to play it safe,
it can be replaced with new/regenerated material (after a dry-air
backfill) once the dewar has been thoroughly "dried out". Aside from one
sieve change (if any) no maintenance should be needed for a year or
more.

***Cooling System***

The constraint on the dewar size (and thus the cooling system) imposed
by the mounting of the f8 secondary behind DECam has been removed: the
f8 will be mounted in front of DECam when in use instead of performing a
top end flip.

The Committee agreed with the project team and with the internal review
conducted in Decemeber 2007 that the proposed cooling system (which
recirculates liquid and gaseous Nitrogen) is technical feasible, and
also agreed on the analysis of the thermal load. The Committee differed
in assessing the size of the thermal margin required and whether the
detector heater power is part of that margin.

The budget appears to be adequate to implement the proposed cooling
system. If the function of this review is only to assess whether the
cost of the proposed solution is realistic, and not to seek cheaper
alternatives meeting specifications, then the following must be taken as
*comments* rather than *recommendations.* There appear to be at least
three technically feasible ways to implement the cooling system:

1.  **Recirculating Nitrogen from a refrigerator off telescope.** This
    will be the most expensive solution and presents some questions
    regarding vibration caused by turbulent flow (as noted by the
    December 2007 internal review), resulting in a need to allocate
    further funds to a prototype. Some of these costs have already been
    expended and the instrument design is based on the assumption this
    will be the solution adopted.

2.  **Parallel JT coolers with gas compressors off telescope.** Based on
    experience from other groups and its own analysis, the Committee
    judged that by connecting several Joule-Thomson (JT) coolers in
    parallel a vibration free, reliable cooling system with relatively
    low operating cost could be constructed at significantly lower
    implementation cost, while maintaining adequate power margin.
    Whether three or four 30 W coolers (e.g., standard Polycold Compact
    Cryocooler with PT30 gas) are required depends on the operating
    margin desired and whether or not an LN2 precool loop is used to
    hasten cool down.

> The Committee noted that the internal review of the cooling system
> conducted in September 2007 primarily addressed the technical
> feasibility of the proposed recirculating Nitrogen system, and did not
> comment on cost effectiveness. However one of the three bullets in the
> executive summary echoed comments from the May 2007 DOE/NSF review by
> stating: "*Consider options which would avoid the use of two-phase
> flow, such as a subcooled LN2 loop **or JT cryocoolers**."*
>
> The project responded that no JT cryocoolers with sufficient capacity
> are available, apparently without considering the use of several in
> parallel.
>
> The preamplifiers inside the dewar dissipate 28W, which was treated as
> a flow into the cooling system, but presumably this heat could be
> conducted to ambient instead of being strongly coupled to the focal
> plane, since the Kapton cables are designed to provide the thermal
> isolation. The project estimated the maximum heat load to be 113W. By
> diversion of the electronic power dissipation, that load could be
> reduced to 85W and by treating the peak detector heater power as
> "safety margin" one can argue that three Polycold coolers sinking 30W
> at a head temperature of 133K will be sufficient. The system could be
> designed so that a fourth head could be installed, if required, while
> still providing a significantly lower cost solution.
>
> The Committee noted that inclusion of a precool loop would not only
> reduce cooling time but would also ensure that the JT coolers rapidly
> reach peak cooling capacity.

3.  **Conventional in situ LN2 tank.** With the decision not to mount
    the f8 secondary behind DECam and perform top end flips, there is
    ample space to allow a conventional Nitrogen tank. Assuming this is
    filled once per day to 65 percent capacity (due to the inclination
    at the prime focus access station) and that a radiation shield
    cooled by boiloff gas also cools the electronics in the vacuum, then
    about

> 46 liters of liquid nitrogen would be consumed per day and tank height
> would be about 14 inches. This would be by far the cheapest solution
> in terms of capital cost but would require CTIO to fund an additional
> \$14K per annum for the liquid Nitrogen (assuming instrument is always
> cold) above the cost of electricity for the current solution, plus the
> labor to fill once per day.

# 3.1.2.2 Recommendations

> 1\. Advance the CCD fabrication work at DALSA/LBNL by earlier
> procurement of lots. This action would not increase costs, but would
> change the fiscal years in which the CCD wafers are procured. 
> Specifically: a) procurement of 15 wafers should be advanced from
> October 2008 to May 2008, resulting in a shift of \$315K, and
>
> b\) procurement of ten wafers should be advanced from October 2009 to
> February 2009, resulting in a shift of \$210K. 
>
> These early actions will serve to retire risk on the CCDs, and thereby
> provide a total of five months additional schedule slack. Money will
> almost certainly be saved through efficiency and yield increases
> obtained by fully engaging the production facilities while they are
> functioning smoothly.
>
> 2\. The Committee recommended CD-3a support for WBS 1.2.
>
> 3\. Make all 2Kx2K CCDs the same thickness so that they will be
> interchangeable; adjust heights of focus CCDs by machining the support
> plate.
>
> 4\. Since the DECam CCDs are significantly affected by cosmic rays,
> evaluate a sample of glass from the C5 melt for potential Potassium-40
> contamination by placing it in front of a DECam CCD and monitoring the
> change in CR event rate. Similarly evaluate a sample of the fiberglass
> used to make the Vacuum Interface Board for potential radioactivity.
> If Beryllium is used (e.g., alloyed with copper) then test this for
> radioactivity too.
>
> 5\. Plan for profilometry of the whole focal plane when cold though a
> flat window; calibrate against a reference flat. The project scientist
> should make sure that those working on the PSF fitting for Weak
> Lensing are shown the data so that they are fully aware of the focus
> variations on all spatial scales, both within and between CCDs.
>
> 6\. Incorporate a small auxiliary valve for slowly bleeding dry air
> into the vacuum during initial pump down to promote outgassing and
> transport of water vapor.
>
> 7\. Consider how to heat the dewar exterior during initial pump down
> to speed up outgassing, though not enough to endanger the dewar
> contents or seals.
>
> 8\. Use of a combination of accessible zeolite getter ***and*** an
> activated carbon getter (which may be inaccessible) in place of the
> ion pump.
>
> 9\. If the ion pump is retained, it is recommend that the team
> directly measure the optical light emitted by it (in photons/s) inside
> the vacuum housing, and then trace the anticipated flux from multiple
> scattering paths that might allow these pump-produced photons to reach
> the CCD focal plane as stray light. (If a convincing case can be made
> that the multi CCD test dewar is a worst case scenario, then an
> empirical test in the multi CCD test dewar may prove more cost
> effective than scattered light analysis.)

# 

# 3.1.3 SISPI and Front End Electronics 

# 

# 3.1.3.1 Findings

The Committee was very impressed by the work of the SISPI and Front End
Electronics (FEE) teams. These teams are comprised of people with
extensive experience on similar projects for the particle physics
community. They have successfully carried this experience over to this
astronomical work while taking advantage wherever possible of existing
tools and hardware in the astronomy community.

## 

## ***FEE Electronics***

-   All issues raised at the May 2007 DOE/NSF review have been
    addressed.

-   The design satisfies the technical performance requirements.

-   The budget for the FEE project is adequate with the exception of
    integration engineering in FY 2010 and FY 2011.

-   The 31 percent contingency for the FEE project looks appropriate
    given that they are building upon an existing design.

-   AlN CD-3a---there is further testing required before proceeding with
    the purchase.

## ***SISPI***

-   Has a well-formulated plan with a believable fully resource-loaded
    schedule.

-   It builds on existing software used at CTIO and by NOAO.

-   Most of the associated hardware are commodity items with well
    understood costs.

-   As such, the contingency of 22 percent is adequate.

-   SISPI is largely a university based software project conducted by
    physicists. It is highly dependent upon base program support. This
    represents a substantial risk that was identified by the Level 2
    managers.

# 3.1.3.2 Comments

***May 2007 DOE/NSF Review Issues***

Project responses to the May 2007 DOE/NSF review recommendations are
shown below (in italics). They showed detailed discussion of the tests
and trade offs involved in making these decisions. The Committee was
satisfied that they made the correct decisions. They showed both PSpice
simulations and test bench results to confirm their choices. For
example, the project chose the AD 8065 preamp, which was vetted by NASA
for extreme temperatures. A detailed discussion of all these choices was
given in their presentations.

*Down-select, before CD-2, the configurations of electronics modules
with justification for each option chosen. Experimental results should
be compared with theoretical expectations.*

***JFET on or off AlN substrate.*** They elected to keep the JFET off
the AlN substrate. There is no performance advantage to keeping it on
the AlN substrate vs. the CCD Interface board. It simplifies the AlN
board design and allows them to repair JFETs in event of a failure.

***Paralleling JFET's versus having a preamplifier board.*** The CCD
Interface Board was designed for parallel JFETs, but allows for
operation of either single or parallel JFETs. Initial testing shows they
require a preamplifier board with either a single or parallel JFET
configuration.

***Preamplifier in or out of dewar (if applicable).*** Preamplifier will
be located inside the dewar. There is no convenient location for the
preamps on the outside of the dewar.

***Single-ended or differential signal transmission.*** Single-ended
signal transmission is final plan. This works well and has met the
readout specifications.

***Kapton versus micro-coax cable.*** They are currently still testing
various options.

***Mechanical design choices of connector attachment to CCD package.***
They are switching from the Nanonics connector to an Airborn 37-pin
nanominiature connector on the CCD package based on the recommendation
of LBNL. LBNL has had good results with this connector and found better
availability. This will be tested with the present AlN substrate.

***Power supply configuration choices.*** A Vicor Switching Power Supply
with a custom filter box will be used in the FE Electronics crate. This
gives performance equal to the linear supplies currently used in the
teststand.

***Multi-crate synchronization choices.*** Synchronizations will be done
in FPGA logic. This will allow a span an expanded range of
synchronization window.

***Consider, as an option, calling an external review of the video
signal chain prior to CD-2.***

An internal review appeared to be extensive and resulted in some useful
design improvements. For example, the ground structure of the Kapton
cable was improved.

***Analyze scenarios that could put CCDs at risk and evaluate mitigation
strategies, if applicable (e.g., power-on or off, or software
initialization transients exceeding safe voltage limits).*** They
presented a detailed analysis of all the possibilities found by their
study, including mitigating designs such as clamping diodes. Details are
available in their presentations.

***List and review all material properties for vacuum compatibility
(e.g., outgassing).*** A detailed table of all materials and their
outgassing rates was shown and did not appear to be a cause for concern.

***Ensure that a failure in one CCD string does not impact the operation
of other CCDs.*** There is a worst case scenario where failure of one
CCD could impact the readout of a second. Unfortunately, due to the
density of buffers required on the boards it is not possible to
completely eliminate this possibility.

***Evaluate tracking errors caused by the unavailability of guiding
information immediately prior to the exposure and resulting from
persistence effects in the guide CCDs**.* An extensive discussion of
this was presented showing a significant difference in signal strength
between guide stars and persistent images in worst case scenarios. This
should not be a problem.

***Other Comments***

It is important to note that the front-end electronics design builds
upon the existing NOAO design. The team has run tests with the Monsoon 1
system that shows that it is close to meeting the needs of the DECAM.
Modifications to the design are needed to change the form factor to
allow the boards to be used in the physical constraints imposed by the
DECAM. Also some components on the earlier boards are obsolete. Finally,
the choice of SLink allows them to leverage existing work from the
particle physics community. The choice of Monsoon allows the use of much
existing firmware and software available from NOAO. This allowed them to
keep costs well under control and to mitigate risk factors. These
choices mean that an overall contingency of 31 percent is appropriate
given the extensive use of existing designs wherever possible and the
successful operation of prototype systems.

A detailed breakdown of the system integration testing schedule was
given. There appears to be sufficient time allocated to debug the system
at Fermilab. There may be a need for additional engineering for the
final integration at Fermilab and CTIO. Analysis of this was complicated
by the fact that engineering resources for this are under the management
and focal plane detector WBS items. Combining EE tasks in WBS Items 1.1,
1.2 and 1.3 shows FY 2007--3.9 FTEs; FY 2008--4.4 FTEs; FY 2009--3.7
FTEs; FY 2010--1.2 FTEs; FY 2011--1.4 FTEs. The FY 2010 and FY 2011
engineering FTEs seem on the low side.

The 34 percent overall contingency for Front End Electronics (33 percent
for R&D/ 36 percent for production) appears to be adequate given that
the present prototypes meet requirements with the addition of some PCB
patches that will be fixed with the next iteration. Much of the
production is an in-kind contribution from Spain and therefore carries
no project contingency.

The decision was made to change the connector on the CCD package from
the Nanonics connector being used to an Airborn 37-pin nanominaiture
connector based on the recommendation of LBNL. This seems reasonable
given LBNL's good experience with the Airborn connector and the
difficulties with availability experienced with the Nanonics connector.
The plan is to try the Airborn connector on three or four units of the
existing v1.0 AlN substrate. After successful results of that test, the
AlN layout will be modified (v2.1) to better accommodate the Airborn
connector. About 20 units will then be fabricated and tested. Successful
completion and satisfactory test results from this small quantity run
will demonstrate readiness for production. A production readiness review
at this stage would be appropriate.

For each board type, they produce twice as many as needed in the final
system and keep the remaining as spares. Thus there are 100 percent
spares planned for each PCB and crate, including the small boards to be
mounted inside the dewar and the Kapton cable. Given that all of the
boards have multiple copies in the system, this should be quite adequate
for swapping out any boards that fail, i.e., there are multiple spare
copies of each board. Assuming that any failed boards will need to be
repaired to keep the whole system operational for up to ten years, an
analysis of which components are most prone to failure and that are
likely to become obsolete should be conducted and an appropriate number
of these component spares should be purchased. Presumably, this would
become part of the in-kind contribution from Spain along with the
production boards.

A detailed presentation was made of the grounding scheme planned for the
DECam. This involves not only the electronics but all the conducting
objects that make up the telescope. The engineer responsible for this
work understands the issues and appears to have a good plan for
eliminating ground loops and noise from unwanted stray currents. There
are several aspects of the existing telescope construction that are not
well understood. Therefore, it is imperative that a planned trip to CTIO
be completed very soon so that a complete plan can be formulated and to
minimize surprises during installation.

The SISPI project has a well-formulated plan with a believable fully
resource-loaded schedule. Since it builds on existing software used at
CTIO and by NOAO and most of the associated hardware are commodity items
with published prices, the overall 22 percent contingency seems
adequate. The purchase of hardware (mostly computers, storage devices,
and network gear) carries an appropriate 20 percent contingency, 10
percent for software licenses, 40 percent for the Cloud Camera, and 30
percent for Fermilab software development. (Note that there was a
clerical error in the contingency numbers originally presented for
1.6.1, leaving out roughly \$28K for two items. This will be corrected.)

Most of the SISPI software development is a university based project
conducted by physicists. This represents a substantial risk to the
project if the university base research program cannot be maintained.
Also, there is no contingency included for this base program work. Any
overrun will have to be covered by scope reduction. This is understood
by the Level 2 managers.

Travel for the Fermilab engineers (primarily for the front-end
electronics) is lumped together in the management section (WBS 1.1). It
is appears to be adequate but it is difficult to determine exactly how
much is planned for the front-end electronics and the SISPI. Likewise,
travel for all the university personnel working on SISPI must be covered
by base program funds. It would be very prudent at this time to make a
detailed estimate of how much travel will be needed for both projects so
that this can be cross-checked against expected funding from the project
and from the base program.

The SISPI resource-loaded schedule, including university personnel,
appears quite reasonable, however, software projects have a history of
slipping. To avoid any unexpected delays, it would be very prudent to
monitor the SISPI progress closely and modify the resource-loaded
schedule as the real rate of software development becomes better
understood.

# 3.1.3.3 Recommendations

1.  Recommend CD-3a approval of AlN production contingent upon the
    planned tests being completed and approved prior to the actual start
    of production.

2.  Identify the components that are most prone to failure and which
    components are likely to become obsolete and use this information to
    determine the appropriate number of component spares.

3.  The MOU with Spain should specify the number of component spares.

4.  Modify the plan to include additional FEE engineering resources in
    FY 2010 and

> FY 2011 for testing and integration.

5.  Schedule a trip, to be taken within six months, by the Power and
    Grounding engineer, to CTIO to work with relevant CTIO engineers and
    transfer knowledge about telescope grounding and local power
    conditions.  

6.  Monitor SISPI progress closely and modify the resource-loaded
    schedule as the real rate of software development becomes better
    understood.

7.  Make a detailed estimate of travel required to complete the project
    at CTIO. This estimate should clearly identify the travel that will
    be funded from the project and the travel that should be funded by
    the base program.

# 

# 3.2 DES Data Management (DESDM) and DECam Simulations

# 

# 3.2.1 Findings

The DES Data Management (DESDM) project has made significant progress
since the May 2007 DOE/NSF review, and has answered a number of the
review recommendations. In particular, the project prepared requirement
documents, included some of the pipelines needed for DES science in the
requirements and WBS, worked to strengthen the group working on the core
astronomical algorithms, and made good progress in defining the
Community Pipeline.

The simulation effort has proven its value to the project, providing the
inputs to the project's data challenges. The enhancements needed to
develop and validate the algorithms are understood by both the DESDM and
simulation teams (examples are the need for more realistic galaxies, and
modelling of the effects of optical misalignment on the delivered PSF).

The third Data Challenge (DC3), begun in October 2007 and completed on
schedule in January 2008, processed about 2.6TB of data, and
demonstrated DESDM's ability to integrate a significant amount of
software into a working pipeline. The scope of the challenge, taking raw
data and populating a database with the derived catalogues was
appropriate. The pipeline made extensive use of trusted astronomy legacy
codes such as SWARP and SExtractor, developed by team member Bertin.

The DC3 processing exposed a number of problems in the software stack
and computational infrastructure such that it took one month to process
ten nights of data.

Timing successful jobs indicated that their standard processing
currently takes about ten CPU-minutes per exposure, although this
excluded preparing and ingesting catalogs into the database. To
understand the time requirements of nightly processing, the Committee
relied on the Figure 9-1of the draft DC3 report (v2) showing processing
times. Since the concurrency of the analysis was unclear the Committee
had some difficulty in matching these processing times to the statement
in Section 9 of the report that the total time to process a night of
data was six hours and ten minutes, excluding catalog ingestion; in the
future, such numbers should be reported in the more appropriate units of
CPU-hours. The team acknowledges that in practice more complex and
time-consuming algorithms will probably be required; indeed, some were
prototyped during DC3. The DC3 report also states that data ingestion is
sped up by a factor of three or four by using a "load and merge"
technique. It would be useful to provide a more complete and detailed
summary of processing times and concurrencies in future reports,
including all steps, and the number of exposures and CCD images
processed per night to get a better understanding of the nightly
processing duration.

The staffing plan is extremely success oriented; this is reflected in
the estimates of the effort required. The DESDM team describes itself as
*lean;* the Committee was concerned that it is too lean. There is
significant risk that the project cannot be successfully completed with
the currently allocated (or proposed for) effort.

The DESDM development plan, notably in the area of astronomy codes,
depends critically on contributions from specific individuals, with
resulting significant schedule risk. If problems occur, the Committee
judged that the current team will be forced to serialise solving them
due to multiple single point failures. The project has made hires to
provide some backup.

However, the Committee suspected that there were several areas of
expertise and responsibility residing with single team members. The
Committee further noted that DESDM has several members who provided less
than 50 percent of their time, as little as 25 percent of their time, to
the project. While it appeared that all team members are making real
contributions to the project, relying on minor fractions of people's
time can be problematic. In addition, the team includes two unspecified
undergraduate students. All of these issues contribute to our conclusion
that the team is understaffed. DESDM appears to lack written plans
and/or processes detailing:

-   Software engineering (requirements at the level that can be coded
    to; a clear development plan; unit- and regression-tests; bug
    tracking; versioning beyond the use of SVN; the build and release
    plans; and code and application documentation)[‏]{dir="rtl"}

-   Error budgets, flowed down from the science requirements

-   Risk management

-   Database specifications, especially for science use

The quality assurance/quality control (QA/QC) process is not well
defined, notwithstanding its successful introduction into DC3. Notably,
the delineation between what QA/QC is done within the astronomy codes
(and simply captured by the QA framework and reported), and what is done
by the framework itself, is poorly defined. The Committee further noted
that the QC portal functions presented by da Costa had some data
discovery functions that would be quite useful for science users of the
DES data.

The Committee was pleased that the weak lensing and difference imaging
(needed for SNe) codes required by some of the science cases were
presented and are partially supported by DESDM. The Committee did not
conduct a detailed review of the functionality present in these codes,
but noted that the individuals responsible have a strong track record in
these areas.

# 

# 3.2.2 Comments

Acceptance testing of DESDM is currently planned at the system level at
the end of the development phase, after the completion of all the Data
Challenges. Phased lower level acceptance testing against individual
requirements, partnered with regression testing, will help to retire
risk during the development phase. It is useful during the development
phase to verify that clearly specified requirements are being met by
functional elements such as code modules, and to re-verify those
requirements are met by later releases of the code modules.

Nightly processing needs have been the major driver to date of the DESDM
design. More emphasis should be given to large-scale reprocessing of
data in driving the design of the database and pipeline technology. The
Committee saw no signs of problems in the current design, but are
concerned that operations such as database ingest may not scale well.

In DC3, five to ten percent of submitted processing jobs failed. It was
not clear that the causes of the failures were known. The draft DC3
report mentions crashes of the PSM module that go away when run a second
time. DC3 did not include non-nominal data (e.g., partial/corrupted
readouts). Future Data Challenges should include such data to test
robustness of the processing systems, as well as the response of the
processing algorithms to pathological data (e.g., first magnitude
stars).

Scaling from nightly processing to annual reprocessing and multi-year
processing for final catalog production is likely significantly to
exacerbate problems with the middleware.

The simulated datasets in DC3 do not include all "real world"
instrumental effects (e.g. residual non-planarity of the focal plane,
other camera and/or telescope error contributions; a turbulent
atmosphere). The Committee would have liked to see system-level, roll-up
of error contributions from camera, telescope, focal plane, electronics
and data reduction to demonstrate that required science performance
(astrometric and photometric) will be met. Such a system-level analysis
of performance should be done in conjunction with the other DES program
elements at the program management level.

The Committee was concerned that the input catalogues for the
simulations did not go deep enough to properly represent the
astronomical noise expected in real images.

The data challenges are carried out on a one-year cycle. While this may
be appropriate for tests at this level, it does not amount to a modern
spiral development pattern. The Committee anticipated that code releases
would be made more frequently than once per year, which would drive the
need for unit and regression tests at the time of the code releases.

The requirements for the database, either in terms of its availability
or functionality, were not clear. Does it need to be up 24/7/365.24?
What sorts of index will be required to deliver acceptable performance
on what class of query?

The Committee noted that the database carries out two different
functions (running the survey and serving up science to the
collaboration and possibly the wider community). It is not clear that
both aspects can be simultaneously optimised.

The DB will be read-only, but the project should consider providing
space for users to generate temporary tables (analogous to the SDSS'
mydb).

Some concern was expressed that the DESDM project assumes that they will
be able to interact with a database from the worker nodes of the HPC
clusters that will be deployed in five years' time. It is not clear how
the objects detected in the nightly processing (and difference imaging)
will be assembled into astronomical sources; this is a tricky problem in
the presence of variable seeing and deblending.

There is significant commonality between web portals for DES science
archive access, the QC access portal, and the described NOAO DPP archive
portal. The project should guard against significant reproduction of
code with common requirements in this respect, and should seriously
investigate a single tool for access to DES, QA, and NOAO (Community
Pipeline) archival data, in collaboration with NOAO DPP.

The QC presentation showed eight FTEs dedicated to this task, a seeming
mismatch with the overall DESDM budget, given the scope of that portion
of the project. The Committee understood that this effort is outside the
direct control of the DESDM, which assesses a significantly smaller
level of effort. Nonetheless, perhaps there is scope for further
integration and task sharing by the QC team with the rest of the DESDM
effort. For example, further exploiting the web portal development done
by the QC team to satisfy other web portal needs.

The DES program is a pathfinder (along with the one degree imager (ODI))
for implementing an instrument-specific community pipeline generating
catalogs for the NOAO E2E system. As a pathfinder, it may require more
than 14 FTE-months of effort and an appropriate contingency should be
assigned.

The Committee did not understand why real data was thought to be harder
to reduce than simulated data.

Mohr stated that the science images would be remapped prior to
difference imaging. The Committee judged that this should be
reconsidered as it results in correlated noise (and problems if the data
is for some reason undersampled).

# 3.2.3 Recommendations

1.  Perform a self-assessment of resource margins and redundancies
    required to achieve DESDM critical functions, to identify principal
    risks (in a severity/likelihood matrix). A risk monitoring and
    mitigation plan should be developed and implemented.

2.  Make a concerted effort to recruit more staff, using the risk
    assessment to identify areas of concern (e.g., astronomy codes,
    databases).

3.  Implement standard software-engineering practices, including
    developing unit-level test requirements for software elements;
    unit-level regression testing of software elements; a bug-tracking
    system for the DESDM systems; and a common code version, build, and
    release system (including the contributed effort from all partners).

4.  Develop a detailed characterization of the computational complexity
    of the time-critical nightly processing steps, including their
    floating-point operation counts and their efficiencies and memory,
    disk and bandwidth loads on target architectures.

5.  Develop contingency plans and resource margins for inevitable
    infrastructure failures.

6.  Investigate the potential for shared development of the DESDM QC web
    portal with the DESDM user/archive portal and NOAO DPP Archive
    portal.

# 3.3 CTIO Facilities Improvement Project (CFIP) and Integration

# 

# 3.3.1 Findings

The CTIO Facilities Improvement Project (CFIP) consists of five separate
improvement projects for the Blanco four-meter telescope in preparation
for the arrival of DECam. These are all projects that CTIO would be
carrying out anyways, but the timing is to some extent driven by DECam
and the DES. The budget for these projects is part of the CTIO base
budget and does not represent a separate allocation.

The largest and most critical of these projects for the DES is the TCS
upgrade project; this is critical for the DES because the existing TCS
cannot drive the telescope at a rate sufficient for the DES cadence.

This first project is underway---about 90 percent of the hardware costs
have already been incurred. The software and electrical engineering team
developed is sufficient for the task as they have experience from the
SOAR TCS. The schedule path is good, with detailed milestones densely
distributed. The target completion date is roughly 12 months prior to
DECam's arrival at the telescope, which appears to be sufficient
contingency.

The TCS timeline has implications for the DECam project; by mid-2009 the
exact DECam command interface to the TCS will be fixed; therefore the
TCS project needs to be nearly complete by then.

The second project is a replacement of the primary mirror radial
supports. This project is well underway. The parts have been ordered,
and, although the machining of the supports have long lead times, the
project appears to be well-positioned to be completed by June
2009---more than one year before the DECam commissioning.

The third project is the construction of a modest (10,000-class) clean
room in the Coude' room of the Blanco. The plans and cost estimates for
this project are underdeveloped, but the proposed room is not a very
demanding project. The timescale for this project's completion is driven
by the needs of another instrument (NEWFIRM) that is scheduled to arrive
at CTIO in 2009. Therefore, the construction of the clean room should
not represent a schedule risk to DES.

The fourth project is a study of the thermal environment of the
four-meter dome. This part of the proposal has no formal budget, and the
output of this study is a report. This is not a significant risk or cost
item. However, this study's findings could lead to potential
improvements that will enhance DES science by improving on the image
quality at the Blanco telescope.

The fifth project is the installation of the utility and cooling lines
that will be needed to support DECam at the Blanco PF. This program is
not well-defined yet, as it is still dependent on the final choice of
the cooling system, and the delivery of the components to CTIO. It is
not a significant cost or schedule driver for the DES.

Effective integration of the TCS replacement project with DECam depends
on the continued role of Tim Abbott as both head of the TCS project and
Deputy Project Manager for DECam.

Integration of the system with the Blanco telescope is defined to be
part of the installation and commissioning phase under the direction of
CTIO, and therefore not part of the DECam project. While this is
understandable from the point of view of the DECam project, this is a
point of vulnerability for DES. The Memorandum of Understanding (MOU0
and supporting documents need to be very clear about not just
responsibilities and control, but costing procedures in case of problems
at this stage.

# 3.3.2 Comments

The TCS replacement project is subject to schedule risks in part due to
other potential commitments of the CTIO staff involved, and cost risks
due to the conversion rate fluctuations. However, the schedule
contingency in the project appears sufficient. As long as the project
remains high priority within CTIO, the DES appears not to be at
significant risk.

The connection between the TCS project and the DECam/SISPI component is
explicitly made; however, the milestones identified are not tied to the
DECam milestones, except by the participation of Tim Abbott.

If the TCS project slips significantly, there needs to be a formal
mechanism for feeding this into the DECam development schedule. In
particular, a document specifying the interlink between the milestones
of the TCS project and DECam/SISPI milestones should be produced.

More detailed plans for the clean room should be developed soon (by the
end of 2008), with the involvement of the DECam team and CTIO, as well
as NOAO/NEWFIRM, to ensure that the cost of the project to CTIO is
controlled.

# 3.3.3 Recommendations

> 1\. Produce a document explicitly linking the TCS milestones that
> affect DECAM/SISPI to the SISPI milestones so that there is no
> slippage risk in the schedules. This should be produced prior to
> January 2009.
>
> 2\. Produce an analysis of the Blanco environment report, with the
> goal of identifying any improvements that can enhance the DES science
> return and impacts on the CTIO budget. This should be done as soon as
> the report is done, as any improvements could feed back into the error
> budget allowed for the other parts of DECam.

**4. PLAN FOR COMMUNITY USE OF DES SYSTEMS,**

**INSTALLATION, COMMISSIONING AND**

**OPERATIONS**

**4.1 Findings**

The Memorandum of Understanding (MOU) between Fermilab, NOAO, and NCSA
represents a significant accomplishment toward ensuring that the DES
System will be delivered as a facility instrument. This MOU defines the
system deliverables to NOAO/CTIO for community use and defines the
procedures for responsibility and control of DECam during installation,
commissioning, and operation.

The MOU forward references several documents that are meant to specify
implementation of agreements agreed upon. These documents are currently
in various stages of completeness.

NOAO has led the development of a process to solicit input from the
astronomical Community on their needs for using DECam. These needs are
being used to define requirements for the DES System that fall outside
those for DES itself. The requirements have been and continue to be
incorporated into the DES system specifications as they develop.

The Committee was presented with a nominal installation plan and
schedule. The committee thought that these plans were adequate for the
current stage of the project. The commissioning and operations plans are
in the preliminary stages of development and are presently at the
conceptual design level. The Committee was presented with a straw man
outline of the commissioning scope and schedule.

The Community Data Pipeline and its data products are a critical part of
making the DEC System a facility instrument and fulfilling the community
needs. Substantial progress was made toward the development of this
pipeline and defining the output data products. The Committee was shown
that existing software code comes close to meeting the current pipeline
specifications.

**4.2 Comments**

The process of fully specifying the Community Data Pipeline is
progressing well. The Committee was encouraged to see that processing
dithered stacks is part of the baseline specifications, as well as
processing individual images. Other special processing modes should be
identified based on common community use cases for the DECam System,
with the goal of providing a final list of modes to be supported by the
end of June 2008.

The Committee agreed that some of the performance specification for the
quality of the data products was too loose. For example, the
specifications state that the photometry derived from the pipeline
should be accurate to ten percent. It is understood that this
specification was driven to accommodate the wide range of possible
photometric spectral bands that could be used by the community. In this
particular instance the Committee judged that a relative specification
would be more appropriate. For example the specification might read:

-   Photometric accuracy/precision shall not be degraded by more than
    approximately one to two percent from the quality of the underlying
    reference data.

-   This would yield better science productivity through increased
    performance that is tied directly to the quality of the reference
    data source.

-   Other data products may be well suited to similar types of relative
    specification.

The scope as presented to the Committee of the DC5 tests on the
Community Data Pipeline was not precisely defined. The specifications of
these tests need to flow down from the requirements defined by the
Community Needs document. This document is scheduled for completion by
mid-2008.

As part of the Community Data pipeline the Committee would like to see
the inclusion of specific data quality assessment products. Where
appropriate these should be taken directly from those developed for the
DES effort. Inclusion of data quality meta-data will improve the
scientific return from the community use of DECam.

Even though the NOAO-led process for defining the community needs of the
DES System has advanced well, the Committee was concerned by the limited
audience that has reviewed the current draft of the Community Needs
document. There is some risk that important requirements for the DES
System may be overlooked thereby limiting the science capability
available to the community. Several examples were discussed at the
review stemming from narrow band imaging use cases, including:

-   Altering the readout timing on the science sensors to lower the read
    out noise so that narrow band imaging is sky limited on long
    exposures.

-   The design of DECam forces the guide sensors to use the same filter
    that is used for the science sensors. This will significantly reduce
    the number of suitable guide stars available when using narrow band
    filters. One possible solution to consider is using the hexapod to
    rotate DECam about the optical axis in order to increase the sky
    area available to the guide sensors. This would help mitigate the
    lower number of suitable guide stars seen through the narrow band
    filter. The project should investigate to what extent the current
    DECam design would support this use case.

-   Another possible solution to the lower guide star flux in narrow
    band filters is altering the guide sensor integration time. The
    impact on telescope guiding performance should be evaluated.

The notional five-week commissioning period that was presented, three
weeks followed by a two-week block, seems optimistic and less than
optimum. The Committee agreed that an initial substantial block of time
is necessary, given the high degree of first time integration occurring
at CTIO. A subsequent series of evenly distributed shorter time blocks
(e.g., four to seven nights) with ample time (e.g., approximately three
weeks) for data evaluation and analysis and any adjustments that are
implied, might be a better strategy.

The Committee recognized that the transition from the integration phase
(on project) to the installation and commissioning phase (CTIO lead) is
critical to the successful implementation of DECam. The Committee was
not presented with a well-defined set of acceptance criteria that would
be required for the hand off. These should be put in place as soon as
possible since the hardware necessary for them would be part of the
DECam project and will have to be budgeted and scheduled.

Because the forward reference documents in the MOU are still in
progress, the specifics remain unclear. Thus what impact these may have
on the DES System design could not be evaluated at the time of the
review.

Included in the set of DECam deliverables are supporting documents
covering user support, technical operation, and maintenance manuals. The
Committee agreed that requirements for these documents need to be
established well in advance of their delivery so that adequate time and
resources are allocated for their production and review by NOAO/CTIO.
These documents are mentioned in writing in the Project Execution Plan,
but should also be explicitly shown in the deliverables table.

**4.3 Recommendations**

1.  Consider making specifications for the Community Data Pipeline and
    its data products more stringent and link them to the quality of the
    underlying reference data where appropriate.

2.  Produce an agreed upon document that explicitly defines the
    Community Data Pipeline deliverables, including its data products,
    by the beginning of 2009.

3.  Develop a broad range of community derived use cases for the DES
    System prior to freezing the implied requirements and specifications
    for the DES System. Evaluate the need for addition requirements for
    the DES System.

4.  Develop a set of technical acceptance criteria that defines the
    transition from integration to installation/commissioning phases on
    the Blanco telescope.

5.  Develop a detailed installation plan based on the current DECam
    design. Use this plan to assess whether there are any design
    implications that need to be incorporated into DECam.

**5. ENVIRONMENT, SAFETY and HEALTH**

**5.1 Findings**

Fermilab has determined that the appropriate level of National
Environmental Policy Act (NEPA) for the proposed action is a Categorical
Exclusion (10CFR1021, Subpart B, Appendix B3.6). The DOE Fermi Site
Office (FSO) approved the Categorical Exclusion on July 11, 2006 and no
further NEPA review is required. The identification of the hazards
associated with the project is documented in the November 6, 2007 DECam
Preliminary Hazard Assessment Document. This document was approved by
the DOE/FSO on November 16, 2007 and served as the basis for the
preparation of January 14, 2008 DECam Preliminary Safety Assessment
Document (PSAD) required for CD-2.

The review of the PSAD indicates that there will be minor or negligible
impact from potential hazards on the health and safety of operating
personnel and negligible impact on the health and safety of all other
Fermilab personnel.

Any potential hazard will be mitigated by insuring that all operating
personnel have received appropriate training and adherence to proper
operational procedures. The DECam constructed has been assessed as a low
hazard activity based on a systematic analysis of worst case accident
scenarios involving risks that would exist if there were no engineered
mitigation actions taken.

The DECam project subsystem managers are responsible for applying
Integrated Safety Management (ISM) principles and functions for their
subsystems in accordance with Fermilab Integrated ES&H Management Plan.
The weekly manager meetings and the bi-weekly project meetings are the
forums where managers raise safety issues related to the DECam project.
When appropriate, advice on safety will be sought from the PPD safety
committees. These committees will be asked to issue Operational
Readiness Clearances (ORC), after appropriate review, for any new major
equipment or facilities constructed by the DECam project. The Project
ORCs are documented and collected in the project's document database.
The DECam ES&H Coordinator plans and coordinates ES&H reviews of the
project and the associated documentation.

**5.2 Comments**

Based upon the ES&H review of DECam project, it is concluded that
construction of DECam will have minor or negligible impact from
potential hazards on the health and safety of operating personnel. It
will have negligible impact on the health and safety of the public and
the environment. The types of processes and materials used by the
project at Fermilab are not new and are covered by existing Fermilab
procedures and practices. Any potential hazard will be mitigated by
insuring that all operating personnel have received appropriate training
and adherence to proper operational procedures.

The ES&H documents on the DECam project were found to be in order at
this stage of the project. However, the PSAD should be revised based on
DOE/FSO and others' suggested comments. During construction and testing
of the DECam project, the DOE/FSO will provide oversight of the project
in accordance with the November 1, 2007 DOE/FSO ISM System Description.

**5.3 Recommendation**

> 1\. Finalize and formally approve the DECam PSAD prior to start of the
> project construction.

**6. COST**

**6.1 Findings**

The DES management presented a DES baseline cost estimate of \$41.4
million (real-year dollars), which is made up of three separate
projects:

-   \$32.9 million DECam funded by DOE

-   \$7.6 million DESDM NSF and other funding

-   \$0.86 million CFIP

The DECam cost estimate can be compared to a CD-1 cost estimate of
\$25.0 million. Cost increases from CD-1 include:

-   Increased equipment costs (\$1.5 million)

-   Incorporated university labor into TPC (\$1.1 million)

-   Incorporated R&D from CD-0 into TPC (\$5.3 million)

The DECam estimate includes contingency of \$5.1 million (32 percent of
remaining cost) on the MIE and \$500K R&D contingency. All costs are
based in FY 2007 dollars, fully-burdened and include DOE/Fermilab
out-year escalation. DECam installation and commissioning effort is not
included in project and HEP base program supports Fermilab scientists.

DECam has requested a CD-3a to release approximately \$2.1 million for
long-lead procurements. This would advance the CCD processing and
packaging in the third quarter 2008 and the hexapod procurement in first
quarter 2009.

The DESDM cost estimate is primarily a software project (heavy labor
component) using a mix of funding sources. DESDM includes software
testing and commissioning, but zero contingency.

CFIP costs are estimated at \$390K for equipment upgrades and \$470K for
labor (\$860K total).

No contingency is included in the DESDM or CFIP portions, although the
CTIO Director holds contingency for CFIP external to the project.

**6.2 Comments**

The DES management team is a capable group. The Committee appreciated
the thorough presentations and frank discussion of the present status
and the challenges that they see ahead. The cost and schedule estimates
for the DECam, DESDM, and CFIP are developed as separate projects. An
integrated project schedule will greatly improve overall project
coordination.

***DECam***

The DECam resource-loaded WBS and schedule are well developed. Estimates
are supported with a documented cost basis. DES appears to be adequately
developed to support CD-2, however cost contingency appears too low
particularly in integration, telescope simulator and activities in
Chile. Cost estimates for CD-3a items (long-lead procurements) are
adequate although some technical issues are unresolved. Approximately
\$2 million of foreign procurements are exposed to currency risk.
Strategies should be considered to mitigate risk.

***DESDM***

The DESDM resource-loaded WBS and schedule were presented based upon
technical judgment. The estimate is primarily labor estimates supporting
a software project. Distributed labor and partial FTEs will require
strong management and coordination

***CFIP***

The CFIP cost and schedule estimate are not resource-loaded. This is
primarily an upgrade to an operating facility.

**6.3 Recommendations**

1.  Reevaluate and update the DECam cost and contingency estimates based
    upon recommendations in this report.

2.  Reevaluate and update the DESDM cost estimates based upon
    recommendations in this report.

3.  Recommend DECam for CD-2 after updating cost and contingency
    estimates.

4.  Recommend DECam for CD-3a after satisfying earlier recommendations
    on CCD's and hexapods.

**7. SCHEDULE and FUNDING**

**7.1 Findings**

The DECam schedule is a logically-linked, resource-loaded schedule
containing approximately 1,000 activities for delivery of the camera to
CTIO. The critical path identified is driven by FY 2010 procurements
(primarily filters, crate cooling system). A CD-4 milestone is
identified in third quarter FY 2012. The overall schedule contingency is
12 months.

The DESDM has a resource-loaded schedule containing approximately 800
activities. The schedule is primarily labor-driven and follows a series
of development cycles. Total DESDM development labor is approximately 44
years. The DESDM is complete with acceptance tests in fourth quarter FY
2011. The overall schedule contingency is approximately nine months.

The CFIP summary schedule was presented---roughly 12 months of schedule
contingency is available. The DECam project presented a
funding-constrained schedule based upon a bottoms-up resource-loaded
schedule. Zero funding is planned for the final year of the project (FY
2012).

MS Project and COBRA (DECam) are the primary scheduling tools chosen to
monitor schedule progress. Monthly Status and Earned Value Reporting
have been in place for five months, and the DOE/Director's Earned Value
Management System (EVMS) review concluded DECam was in compliance for
self certification.

DESDM funding is provided via a proposal submitted to NSF for \$2.94
million to support the project scope. CFIP infrastructure improvements
will be provided using operating funds. The funding profiles for each of
the three subprojects are shown in the Table 7-1.

**7.2 Comments**

The Committee judged the schedule maturity for the DES projects to be at
the appropriate level to support CD-2. Schedule contingency of 12 months
(DECam) and nine months (DESDM) is considered adequate.

Schedule contingency is not costed. Any schedule delays will draw from
contingency and should be estimated by the project for planning
purposes.

Although the project has identified the primary project activities,
overall integration should be improved. An integrated DES project
schedule with identified subproject linkages will facilitate
coordination of the entire DES project.

The DECam obligation profile is front-loaded and early use of
contingency will need to be carefully managed. The long-lead
procurements are important to help retire risk.

DECam project team should work closely with Fermilab to minimize
potential continuing resolution effects on the project's procurement
plans. Fermilab's Project Management Group can play an important role in
informing laboratory management of any schedule or funding issues.

There is an effective use of milestones on all projects to monitor
overall progress--- commendable. Each project has identified the
critical path activities and is actively managing schedule contingency.
FTE estimates or manpower profiles provided for most areas of the DES
project.

**7.3 Recommendations**

1.  Develop an integrated DES project schedule that effectively connects
    the three projects.

2.  Recommend DECam for CD-2 and CD-3a.

**8. PROJECT MANAGEMENT**

**8.1 Findings**

The DES project is made up of three major subprojects, the DECam
project, the DESDM project, and the CFIP. A project director oversees
the DES Collaboration and project that currently consists of 12
participating institutions and over 100 individual collaborators. The
Office of High Energy Physics within DOE/SC funds the majority of the
DECam project. Fermilab is the lead institution for DECam. The National
Center for Supercomputing Applications (NCSA) at the University of
Illinois at Urbana-Champaign (UIUC) leads the Data Management project
with NSF funding. The NOAO leads the CFIP with funding from the NSF as
well. In-kind contributions and no-cost resources and deliverables form
substantial and critical contributions in all three projects. In
addition to the American collaborating institutions, there are three
foreign consortia: United Kingdom-DES, Spain-DES, and Brazil-DES, each
with specific in-kind deliverables. Additionally Spain-DES and the
University College of London (UCL) have successfully applied for and
received explicit funding from their respective funding agencies to
participate in the DES project subject to U.S. funding agency approval
of the project. In several instances, the in-kind or no cost
contributions from collaborators include subcontracts to other
institutions (for example University of Michigan subcontracts to Bonn
University for the shutter) or vendors (UCL procures the optics from
potential multiple suppliers).

A DES Council representing Fermilab, NCSA, and NOAO provides director
level oversight of the project. H. Montgomery (Fermilab), R. Crutcher
(NCSA), and T Boroson (NOAO) presently serve as its members. A final
draft global MOU, known as *The Understanding*, was developed concerning
the DES Council and its working relationship between the three principal
institutions with latest revisions having been made shortly before this
review (January 24, 2008). It should be ready for final signatures
imminently.

Each of the DES collaborating institutions has a letter of application
outlining its commitments and in-kind contributions to the DES project.
The DES project has been actively transforming these letters of
application into MOUs, and the majority of them have been signed with
annually revised statements of work.

The governance of the DES project reflects its collaborative
multi-institution and multi-agency aspects. The DES Management Committee
represents the interests of the collaboration. The DES Project Director
(J. Peoples) chairs this committee with members elected from the
collaborating institutions and consortia with the DECam Project Manager
(B. Flaugher), the DES DM Project Leader (J. Mohr), and CTIO Director
(A. Walker) participating as ex-officio members. The management
committee addresses collaboration management and overall DES project
coordination.

The DES project has a Science Committee that is tasked with providing
the survey science requirements to the DES project as a whole. A
Community Needs Working Group was established to review and integrate
community needs into the three projects. It is the forum for the
integration of the community pipeline into the NOAO/DPP E2E system and
the DECam use of the NOAO DTS.

Each of the major projects within the DES project (DECam, DESDM, and
CFIP) have project managers and project offices commensurate with their
relative size.

The DECam project is the largest of the three. It has added a 50 percent
Project Scientist with systems and scientific requirements
responsibility since the May 2007 DOE/NSF review. The project scientist
joins the project manager and two deputies that previously formed the
DECam Project Office. One of the two deputies is a member of the CTIO
staff and serves both as the principal CTIO contact for DECam and is
responsible for integration and testing plans and execution of DECam at
CTIO. The DECam project has six Level 2 Managers each responsible for
specific major subsystems or activities within the DECam project.

The DESDM project has a project director, project manager, and five WBS
Level 2 activities. The CFIP is organized within the CTIO as its
collection of projects drawing upon the existing operating budget of
CTIO for funding and CTIO staff for resources.

The highest level of integrating systems support within the DES project
is provided by the DES Project Director and an executive committee
constituted by the project offices of the three projects that meets on
an approximately monthly basis.

The DES project and the DECam project in particular (with regard to DOE
requirements) fully responded to eight of the nine management
recommendations from the May 2007 DOE/NSF review with recommendation on
systems and integration support at the DES project level still being
examined.

Part of the charge of this review was to assess the readiness of the
DECam project to receive CD-2/3a, (Approve Performance Baseline and Long
Lead Procurement). The specific areas within DECam seeking long lead
procurement approval include WBS 1.2 CCD Processing, WBS 1.2 & 1.3 CCD
Packaging, and WBS 1.5 Hexapod procurement.

The DECam project presented a complete and adequate set of project
documentation necessary as part of establishing the project baseline and
receiving CD-2. Appendix F summarizes the lines of inquiry (LOI) that
were pursued as part of this review. The DECam project responses to all
17 LOI were deemed satisfactory with only four LOI having any additional
comment. These four were risk management, start-up test plan, system
functions and requirements, and the Integrated Project Team.

**8.2 Comments**

The DECam project has adequately prepared all the requirements necessary
to meet the requirements for CD-2/3a. The Committee judged that the
documentation presented by the DECam project as required by the External
Independent Review (EIR) summary assessment and the LOIs were
satisfactory and adequate for DOE to approve CD-2/3a for the DECam
project (see Appendix F). The Committee endorses DOE approval of CD-2
for the DECam project once the cost and contingency baselines have been
reevaluated and revised (as necessary) for the telescope simulator
activities and tests, DECam integration and assembly tasks, all tasks
and activities at CTIO, and incorporated a detailed estimate of travel
paid by the DECam project vs. operating or base funding (cf. §3.1.3.3
and §6). The Committee endorsed DOE approval of CD-3a, long-lead
procurement authorization for the CCD production (cf. §3.1.2). The
Committee also endorsed CD-3a for the front end electronics AIN
production once a design review and noise tests have been completed of
the AINv2.1 (cf. §3.1.3.3). The Committee endorsed CD-3a and procurement
of the hexapod assembly once a finite element modal analysis of DECam
mounted in the telescope top ring needed to validate the design of the
cage, vanes, and hexapod is completed (cf. §3.1.1).

The DESDM and CFIP projects presented their project execution plans
(PEP). The Committee judged these to be adequate initial plans. The
DESDM project has a good project team in place and acknowledges that
their proposed project baseline is not consistent with the scope of work
activities anticipated. The Committee specifically identified areas
where DESDM management can strengthen the effort as detailed in Section
3.2. The DESDM baseline change control thresholds and protocols as
contained within its PEP (Section 4.9.2) appeared to be inconsistent
with respect to management levels and schedule delays. Additionally, the
schedule delays are not referenced to either explicit level milestones
or tasks.

Integration of the three individual projects within DES remains an area
where additional effort will bring substantial benefit to the
collaboration. Milestones, schedules, and risk and issues management
should be carefully synchronized and integrated so as to minimize delays
or increased costs stemming from inadequate coordination.

*The Understanding* (MOU) is a very comprehensive and strong document.
It should be signed as soon as possible while understanding that
revisions will likely be inevitable. Of necessity for keeping a
tractably sized document, *The Understanding* references many subsidiary
documents. Most of these documents do not yet exist or are only in
preliminary draft form. An explicit schedule should be developed for the
subsidiary documents referred to within *the Understanding*.

Only a general statement of support by Fermilab to the DECam
installation after CD-4 is included in *The Understanding*. This creates
some concerns and uncertainties. It is true that support for operations
and commissioning have generally been forthcoming on other projects, and
the Committee judged that Fermilab will certainly support DECam
installation and commissioning activities after CD-4. Nonetheless, an
explicit declaration of the minimum level of support that DES can
anticipate from Fermilab would mitigate uncertainties for both CTIO and
DES.

Recent restructuring and budget uncertainties within the United Kingdom
Science and Technology Facilities Council (STFC) resulted in a complete
reprioritization process of funding in the UK. This prioritization
process has left the status of UK funding for the DECam optics polishing
in question. The DOE, NSF, and DES project should work together to
minimize and mitigate this particular risk.

The DES project and the DECam project in particular have worked and
improved configuration, systems, and integration management since the
May 2007 DOE/NSF review. DECam has developed several documents, and
management plans connected within this area including an initial draft
DECam systems configuration list and interface summary currently under
development. While a great improvement, the addition of a 50 percent
project scientist is not sufficient to address all the configuration,
systems, and integration issues and risks facing DECam or DES. These
areas are weak at this time and need to be strengthened within DES and
DECam. Failure to address these deficiencies will increase risk of
unrecoverable cost increases and schedule delays. Examples of items that
are lacking or only in most preliminary nascent state are configuration
controls, requirement flow downs and traceability, and error budgets and
trees.

Risk and risk management were areas of concern for the committee. The
DES project showed a cognizance of many of the risks associated with the
project as a whole. DECam has written a risk management plan and
developed an initial risk register. Likewise DESDM and CFIP have shown
awareness of risks associated with their respective projects. All of
this is good as far as it goes, but the Committee was concerned that it
is not nearly adequate enough. The DECam risk registry does not include
any summary or integrating risks and is not used as a management tool to
drive behavior or attention within the project. Risk and issues logs
routinely discussed, reviewed, updated and acted upon are simply not
used at present. There is no explicit effort at the DES project level of
highlighting, tracking, and addressing mitigation strategies for risks
and issues. The DES project, DECam, DES DM, and CFIP should individually
and collectively develop and implement risk and issue management
approaches that are used on a day-to-day management basis. A conscious,
concerted effort in this area will reap benefits in a series of projects
that execute more smoothly and predictably and integrate more easily.

In the area of risk management and mitigation, the DES Collaboration
needs to look at mitigating risks during operation and science data
taking, as well as during the project execution phase. The optical
filters are one example of an area where operational risk reduction can
be done at the present time. When soliciting quotations from the
potential filter manufacturers, options to procure additional spare
filters at the time of the initial order should be requested. Additional
filters purchased at the same time as the initial set of filters will
likely cost only a fraction of what the initial filter set will cost.
The filters will be removed and exchanged during the course of
operations and will have a real risk of being damaged or broken. Having
operational spares purchased at the time of the procurement of the
initial set will significantly reduce the down time or data acquisition
interruption during operations.

Another risk mitigation strategy that has been mentioned previously in
this report (cf. §3.1.2) is accelerating the CCD production rate. The
major risk here is the *non-statistical* incidents that could disrupt
production or result in the loss of complete or substantial fractions of
lots of wafers/chips. The DECam project should allocate additional
funding for the CCD manufacturing in order to accelerate the production
rate to that approaching the maximum that the facilities (LBNL/DALSA)
can handle. The limit on the production rate should be the capacity of
LBNL/DALSA and not a funding limit. *The DECam project should do this
accelerated CCD manufacturing rate even if it results in the slowing of
other activities within the DECam project to accommodate the revised
funding allocation.* CCD production was identified as one of the major
critical risk areas of the DECam project and moving to a configuration
where it is capacity limited rather then production limited has several
benefits. It retires early the risk on the CCDs. It will provide
additional schedule buffer or insurance. It will likely reduce costs as
it will increase efficiency and yield and learning curve benefits in
processing will be realized.

The DES Project Director continues to do a remarkable job in assembling
an enthusiastic collaboration and bringing a diverse set of
complementary capabilities with significant in-kind contributions
together. The DES Collaboration is an enthusiastic competent team and
the Project Director's direct efforts have greatly increased the
likelihood of success of the overall effort and reduced the overall
risk.

**8.3 Recommendations**

1.  Proceed with obtaining CD-2 for DECam project once recommended
    cost/schedule modifications have been incorporated into the project
    baseline.

2.  Proceed with obtaining CD-3a for DECam project, consistent with the
    recommendations in section 3.1.

3.  Implement active explicit risk/issues management on a day-to-day
    basis across the DES project (DECam, DESDM, CFIP) before the next
    DOE/NSF review.

4.  Provide additional explicit configuration, integration/systems
    effort within DES and DECam before the next DOE/NSF review.

5.  Codify an explicit minimum level of support the DES can anticipate
    from Fermilab for DECam installation post CD-4 before the next
    DOE/NSF review.

APPENDIX A

**CHARGE TO THE**

**COMMITTEE**

***U.S. Department of Energy***

***and the***

***National Science Foundation***

November 15, 2007

[To:]{.underline} Mr. Daniel Lehman, Director, DOE Office of Project
Assessment, SC-28

[Subject]{.underline}: Request to Review the Dark Energy Survey project

The Department of Energy (DOE) Office of High Energy Physics (OHEP) and
the National Science Foundation (NSF) request that you conduct a review
of the preliminary design of the entire Dark Energy Survey (DES) project
on January 29 -- 31, 2008 at Fermi National Accelerator Laboratory
(FNAL).

The DES experiment plans to utilize the existing Blanco Telescope at the
Cerra-Tololo Inter American Observatory (CTIO) in Chile to study the
nature of dark energy. The project includes the fabrication of a new
camera (DECam) optimized for the study of dark energy, the CTIO
facilities improvement project (CFIP) for upgrade to the telescope and
CTIO facility, and a data management system (DESDM). The DES project
plan calls for funding for the DECam to be provided by DOE and funding
for the DESDM and CFIP from NSF. Funding for CFIP would be provided
through the NOAO, CTIO's parent organization. In addition, funds are
being requested from other U.S. and foreign institutions.

For DOE, this review will serve to assess the project's readiness to
establish the technical, cost, and schedule baseline of the Dark Energy
Camera (DECam). For NSF, the review will serve to assess the progress
and status of the proposed CFIP and DESDM. The validation of the DECam
baseline is needed for Critical Decision 2, Approval of Performance
Baseline. In addition, OHEP is planning to seek CD-3a (Approve Limited
Fabrication) so that the DECam may begin long lead procurements and a
limited set of other fabrication activities that are needed to maintain
the proposed schedule with adequate contingency. Please review the
necessity of and the project's readiness to carry out these activities.

In carrying out its charge, the review committee is requested to
consider the following questions for all parts of the project:

1.  Technical: Does the technical design and associated implementation
    approach for each part of the experiment satisfy the scientific and
    technical performance requirements?

2.  Cost: Is the cost estimate complete, well documented, and credible?
    Is the contingency adequate for the risk?

3.  Schedule: Is the proposed schedule reasonable and appropriate in
    view of the technical tasks and projected funding profiles?

4.  Management: Is the management structure adequate to deliver the
    proposed technical scope within specifications, budget and schedule?

5.  Limited Fabrication: Are the requested long-lead procurements and
    other fabrication activities scheduled as part of CD-3a necessary to
    achieve the stated schedule? Have the necessary preparations to
    execute these activities?

6.  Are the environmental, safety and health (ES&H) aspects being
    properly addressed given the project's current stage of development?

7.  Documentation: Is the DECam documentation required by DOE Order
    413.3A for CD-2 and CD-3a complete? Does the project satisfy all 16
    lines-of-inquiry specified in DOE O 413.3A (see Appendix A)?

Kathleen Turner is the DOE program manager for the Dark Energy Survey
and will serve as the Office of High Energy Physics contact for the
review. Nigel Sharp of the NSF Division of Astronomical Sciences will
serve as the NSF contact for the review.

We appreciate your assistance in this matter. As you know, these reviews
play an important role in our programs. Both agencies look forward to
receiving the committee's report, within 60 days of the review.

/s/ /s/

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

Wayne Van Citters Dennis Kovar

Director Associate Director

Division of Astronomical Sciences for High Energy Physics

National Science Foundation Office of Science

Arlington, VA Department of Energy

Germantown, MD

APPENDIX B

**REVIEW**

**PARTICIPANTS**

**Department of Energy/National Science Foundation Review of the**

**Dark Energy Survey (DES) Project**

**REVIEW COMMITTEE PARTICIPANTS**

## **[Department of Energy]{.underline}**

Daniel R. Lehman, DOE/SC, Chair

**[Review Committee]{.underline}**

Andy Albrecht, UC Davis

Julian Borrill, NERSC

Bryan Butler, NRAO

Rob Cameron, SLAC

Pepin Carolan, DOE/FSO

Chuck Claver, NOAO

Ian Dell'antonio, Brown U.

Sat Goel, DOE/SC

Alex Grillo, UCSC

Matt Johns, Carnegie Inst.

Robert Lupton, Princeton U.

Paul Padley, Rice University

Mark Reichanadter, SLAC

George Ricker, MIT

Kem Robinson, LBNL

Roger Smith, CalTech

Nick Suntzeff, TAMU

### **[Observers]{.underline}**

Mike Procario, DOE/SC

Kathy Turner, DOE/SC

Paul Philp, DOE/CH

Nigel Sharp, NSF

Tom Barnes, NSF

**APPENDIX C**

**REVIEW**

**AGENDA**

**Department of Energy/National Science Foundation Review of the**

**Dark Energy Survey (DES) Project**

**AGENDA**

**[Tuesday, January 29, 2008---Comitium, WH2SE]{.underline}**

8:00 am DOE Executive Session D. Lehman

9:00 am Welcome/Laboratory Overview---**One West**, **WH1W** H.
Montgomery

9:15 am DES Overview -- Science Intro, DES Org, MOU Structure J. Peoples

9:55 am DECam Technical Requirements/Comparison of D. DePoy

DECam/DES to Other Projects

10:25 am BREAK

10:40 am DECam Overview Including Integration at CTIO B. Flaugher

11:15 am DES/DM Overview and Requirements J. Mohr

11:45 am Roles of NOAO A. Walker

12:15 pm LUNCH

**Parallel Plenary Presentations**

***DECam---One West, WH1W***

1:15 pm DECam Project Mechanics (MOUs, Config. Control, Doc. Summ.) W.
Merritt

1:30 pm CCDs T. Diehl

1:50 pm Front End Electronics T. Shaw

2:10 pm Survey Image System Process Integration (SISPI) J. Thaler

2:25 pm Opto-Mechanical System A. Stefanik

2:45 pm Integration B. Flaugher

***DES Data Management System---Hornets Nest, WH8X***

1:15 pm Cost, Schedule C. Beldica

1:40 pm Archive D. Cai

2:00 pm Pipeline Processing Framework G. Daues

2:20 pm Astronomy Processing J. Mohr

2:40 pm Photometric Calibration D. Tucker

3:00 pm BREAK

3:15 pm Breakout Sessions

***DECam***

CCDs---**SiDet** T. Diehl/J. Estrada

Front End Electronics/SISPI---**Black Hole, WH2NW** T. Shaw/J. Thaler

Opto-Mechanical/Optics---**Snake Pit, WH2NE** A. Stefanik/P. Doel

***Status DESDM Development***---**Hornets Nest, WH8X** J. Mohr

DECam Simulations Program

DESDM Data Challenges

DESDM Requirements and Technical Specifications Overview

4:45 pm DOE Executive Session

6:00 pm Adjourn

**[Wednesday, January 30, 2008]{.underline}**

8:00 am Breakout Sessions

***DECam***

CCDs---**RaceTrack**, **WH7X** T. Diehl/J. Estrada

Front End Electronics/SISPI---**Black Hole, WH2NW** T. Shaw/J. Thaler

Opto-Mechanical/Optics---**Snake Pit, WH2NE** A. Stefanik/P. Doel

Cost, Schedule, and Management---**Comitium, WH2SE** B. Flaugher

***Data Management***

Current Issues/View toward Operations-**One North, WH1NW** J. Mohr

11:00 am Data Management-Cost, Schedule and Mgmt---**Comitium, WH2SE**
C. Beldica

***CFIP***

CFIP Tech. Details, Cost/Sched.---**Hornets Nest, WH8X** T. Abbott

of the Telescope Control System

12:00 pm LUNCH

1:00 pm ***DES Mgmt Session***

DECam as a Community Instrument---**Comitium, WH2SE**

Community Data Management, incl. Community Pipeline req. C. Smith

Community Pipeline Implementation Plan J. Mohr

Installation, Commissioning and Operations Arrangement A. Walker

Management Questions/Discussion (starting at 2:30)

DES/FNAL/NCSA/NOAO MOU

Institutional MOU

Answer questions from the Committee from the previous day

3:00 pm DOE Executive Session---**Comitium, WH2SE**

**[Thursday, January 31, 2008]{.underline}**

8:00 am Committee Working Session

9:30 am Executive Session Dry Run---**Comitium, WH2SE**

12:00 pm Lunch

1:00 pm Closeout Briefing with DES Management---**Curia II, WH2SW**

2:00 pm Adjourn

**APPENDIX D**

**COST, SCHEDULE**

**and**

**FUNDING TABLES**

**Cost and Schedule**

The CD-1 DOE Total Project Cost (TPC) range was \$24.1 - \$26.7M for the
DECam Project.

DOE TPC presented at this Review was \$32.9M, including 32% contingency
on Major Item of Equipment (MIE) funds.

In addition, in-kind contributions to the DECam Project are estimated at
\$6.5M.

Revisions to the DOE TPC were:

-   Technical cost increases since May CD-1 Review: \$1.5M

-   DOE funded University Base Grants (UIUC): \$1.1M

-   R&D from CD-0 to August 1, 2007: \$5.3M

DECam Project Funding Profile:

![](media/image1.wmf){width="7.796456692913385in" height="1.55in"}

**DECam Schedule Summary**

![](media/image2.wmf){width="8.5in" height="6.192361111111111in"}

**Milestones**

Proposed/actual Critical Decision (CD) Milestones:

  ----------- --------------------------------------- -------------------
  Milestone   Description                             Baseline Date

  0.0         CD-0: Approve Mission Need              November 22, 2005
              (Ground-Based Dark Energy Experiment)   

  0.1         CD-1: Approve Alternative Selection and October 10, 2007
              Cost Range                              

  0.2         CD-2: Approve Performance Baseline      2nd Qtr. FY 2008

  0.3a        CD-3a: Approve Limited Construction     2nd Qtr. FY 2008

  0.3         CD-3b: Approve Start of Full            1st Qtr. FY 2009
              Construction                            

  0.4         CD-4: Approve Project Completion        3rd Qtr. FY 2012
  ----------- --------------------------------------- -------------------

CD-3a approval is being requested for long-lead procurements of: charge
coupled device (CCD) processing and packaging and hexapod procurement.

**APPENDIX E**

**MANAGEMENT**

**CHART**

**DES Organization---A Collaboration Perspective**

![](media/image3.jpeg){width="9.270833333333334in" height="4.375in"}

**DECam Project Level 2 Managers**

![](media/image4.jpeg){width="8.96875in" height="5.083333333333333in"}

**APPENDIX F**

**EIR LINES of**

**INQUIRY**

**EIR Summary Assessment of the DES/DECam Project Performance Baseline**

+-----------+--------+-------------------------------------------------+
| **EIR     | **SC   | **Comment**                                     |
| Element** | Review |                                                 |
|           | Team   |                                                 |
|           | Assess |                                                 |
|           | ment** |                                                 |
+===========+========+=================================================+
| 1\. Work  | Satisf | **Project Response**: The WBS is developed to a |
| Breakdown | actory | level of detail appropriate for each individual |
| Structure |        | subsystem. Typically it goes to level 4 or 5.   |
|           |        | The WBS dictionary has been developed and       |
|           |        | detailed basis of estimate documentation has    |
|           |        | been prepared where appropriate. The notes      |
|           |        | field of each task with M&S or Labor cost over  |
|           |        | \$10k and all the milestones contain a          |
|           |        | description of the scope of the task, equipment |
|           |        | and/or labor basis of estimate. There is a      |
|           |        | column in the schedule file called BOE which    |
|           |        | contains the document number in the DES         |
|           |        | document data base for the basis of estimate    |
|           |        | information or it indicates that the relevant   |
|           |        | information in contained in the notes field.    |
|           |        | There is also a column called BOE hyperlink     |
|           |        | which contains a link directly to the BOE       |
|           |        | documents.                                      |
|           |        |                                                 |
|           |        | **Committee Response**: Committee concurs.      |
+-----------+--------+-------------------------------------------------+
| 2\.       | Satisf | **Project Response**: A Resource Loaded         |
| Project   | actory | Schedule for FY07-FY12 has been developed using |
| Cost and  |        | Microsoft Project. Level 2 managers and Cost    |
| Resource  |        | Account Managers are identified in the MSP      |
| Loaded    |        | file. The cost basis for the TPC is currently a |
| Schedule  |        | combination of responses from vendors,          |
|           |        | engineering estimates and estimates from        |
|           |        | previous experience building prototypes or      |
|           |        | similar projects. The notes field in the MSP    |
|           |        | file contains the basis of estimate for each    |
|           |        | task with M&S or Labor cost over \$10k as well  |
|           |        | as many of the smaller tasks. A WBS dictionary  |
|           |        | produced from the MSP file contains a printout  |
|           |        | of the entire schedule, including the notes     |
|           |        | field and the resources on each task; this      |
|           |        | information is provided in PDF format along     |
|           |        | with the MSP file.                              |
|           |        |                                                 |
|           |        | The deliverables and the definition of project  |
|           |        | completion are described in the DECam Project   |
|           |        | Execution Plan. Specific expected performance   |
|           |        | parameters for the DECam Project are contained  |
|           |        | in the DECam Technical Specifications and       |
|           |        | Requirements document and in the DECam Project  |
|           |        | Technical Design Report.                        |
|           |        |                                                 |
|           |        | The Resource Loaded Schedule is matched to the  |
|           |        | funding profile provided by FNAL.               |
|           |        |                                                 |
|           |        | **Committee Response**: Committee concurs, with |
|           |        | updates to cost estimate as recommended in the  |
|           |        | report.                                         |
+-----------+--------+-------------------------------------------------+

+-----------+-------+-------------------------------------------------+
| **EIR     | **SC  | **Comment**                                     |
| Element** | R     |                                                 |
|           | eview |                                                 |
|           | Team  |                                                 |
|           | As    |                                                 |
|           | sessm |                                                 |
|           | ent** |                                                 |
+===========+=======+=================================================+
| 3\. Key   | Sa    | **Project Response**: The CD-2/3a review is     |
| Project   | tisfa | scheduled for Jan. 29^th^. For our planning     |
| Cost and  | ctory | purposes we have assumed that the CD2/3a        |
| Schedule  |       | approval will be granted on April 1^st^, 2008.  |
| As        |       | We are requesting CD3a approval at the Jan.     |
| sumptions |       | 29^th^ review for the CCD processing, the CCD   |
|           |       | packaging and testing and for the Hexapod.      |
|           |       | These are all long lead time and early          |
|           |       | procurements will reduce the schedule risk to   |
|           |       | the project.                                    |
|           |       |                                                 |
|           |       | We are estimating that the CD3b review will be  |
|           |       | in July or August and the full CD3 approval     |
|           |       | would be granted on Jan.1, 2009. Following      |
|           |       | DOE-OHEP guidance, scientist effort on the      |
|           |       | project is entered into the schedule file and   |
|           |       | tracked at zero cost. Technical labor and       |
|           |       | procurements are costed resources. Earned Value |
|           |       | and monthly reporting started on August 1^st^,  |
|           |       | 2007, 2 months prior to CD1 approval.           |
|           |       |                                                 |
|           |       | The project also provided an estimate of the    |
|           |       | cost of the R&D specific to the DECam project   |
|           |       | that occurred between CD-0 and CD-1 as well as  |
|           |       | an estimate of the cost of engineering          |
|           |       | supported by the DOE University base grant.     |
|           |       | Only UIUC is supporting technical labor from    |
|           |       | their base grant. This will be included in the  |
|           |       | project TPC. CD-0 Approval for a Ground-Based   |
|           |       | Dark Energy Experiment was approved on November |
|           |       | 22, 2005. Overhead rates are provided by the    |
|           |       | Fermilab accounting office. The escalation      |
|           |       | rates come from the annual guidance from        |
|           |       | DOE-OHEP for the Laboratory\'s budget           |
|           |       | submission. The MSP file contains the direct    |
|           |       | costs for M&S and Labor. The escalation rates   |
|           |       | and overhead are applied by the COBRA program.  |
|           |       |                                                 |
|           |       | A contingency factor is set by the L2 managers  |
|           |       | on each task based on the guidance:             |
|           |       |                                                 |
|           |       | -   10% - similar devices fabricated and tested |
|           |       |                                                 |
|           |       | -   20% - have not yet completed fabrication    |
|           |       |     and proved that it works                    |
|           |       |                                                 |
|           |       | -   30% - have a design and experience with     |
|           |       |     similar devices and/or initial vendor       |
|           |       |     discussions but no quote                    |
|           |       |                                                 |
|           |       | -   40% - have a design but no vendor input     |
|           |       |     and/or little experience with similar       |
|           |       |     devices                                     |
|           |       |                                                 |
|           |       | -   50% - complete guess or large variation in  |
|           |       |     vendor quotes                               |
|           |       |                                                 |
|           |       | The CCDs receive special treatment:             |
|           |       |                                                 |
|           |       | -   The base schedule includes 6 Lots of CCDs   |
|           |       |     (2A-F). This will provide 126 wafers minus  |
|           |       |     breakage. With our current estimate of the  |
|           |       |     yield 100 wafers =106 ± 28 good devices.    |
|           |       |                                                 |
|           |       | -   An additional lot is included in the cost   |
|           |       |     contingency to cover a yield as low as 15%. |
|           |       |                                                 |
|           |       | -   The processing, packaging and testing of    |
|           |       |     the contingency lot would occur in FY10 so  |
|           |       |     the contingency is placed in this year. The |
|           |       |     tasks are in the schedule to keep track of  |
|           |       |     the schedule impact.                        |
|           |       |                                                 |
|           |       | -   Schedule risk due to low CCD yield is       |
|           |       |     mitigated by the ability to install CCDs at |
|           |       |     CTIO.                                       |
|           |       |                                                 |
|           |       | The contingency factors were reviewed by        |
|           |       | project management and updated to reflect       |
|           |       | recent vendor responses to requests for         |
|           |       | information and requests for quotes. This       |
|           |       | analysis produced a 32% contingency on the MIE  |
|           |       | which is reflected in the TPC. The Baseline     |
|           |       | Schedule includes 12 months (33%) of schedule   |
|           |       | contingency.                                    |
|           |       |                                                 |
|           |       | **Committee Response**: Committee concurs, with |
|           |       | updates to contingency assessment as            |
|           |       | recommended in the report.                      |
+-----------+-------+-------------------------------------------------+

+------------+-------+------------------------------------------------+
| **EIR      | **SC  | **Comment**                                    |
| Element**  | R     |                                                |
|            | eview |                                                |
|            | Team  |                                                |
|            | As    |                                                |
|            | sessm |                                                |
|            | ent** |                                                |
+============+=======+================================================+
| 4\.        | Sa    | **Project Response**: The Critical Path (tasks |
| Critical   | tisfa | with 0 total slack) for the project is         |
| Path       | ctory | currently driven by funding constraints in     |
|            |       | FY08 and FY09 which push procurement of the    |
|            |       | major systems (filters, cooling for the FEE    |
|            |       | crates) into FY10. The next closest tasks to   |
|            |       | the critical path (Design and test of the LN2  |
|            |       | cooling system) have roughly 3 weeks of slack. |
|            |       | These tasks are on schedule and are            |
|            |       | technically driven. The schedule durations are |
|            |       | integrated in to the MSP schedule file and are |
|            |       | based on engineering estimates and vendor      |
|            |       | quotes and on experience with the design and   |
|            |       | fabrication of prototypes.                     |
|            |       |                                                |
|            |       | **Committee Response**: Committee concurs.     |
+------------+-------+------------------------------------------------+
| 5\. Risk   | Sat   | **Project Response**: The DECam Risk           |
| Management | isfac | Management Plan exists and is in active use.   |
|            | tory, | It includes documentation, analysis, and       |
|            | with  | mitigation. Risks have been identified and     |
|            | co    | classified according to a combination of       |
|            | mment | likelihood and impact. Mitigation plans have   |
|            |       | been developed down to WBS Level 3 for all     |
|            |       | moderate or high risks and status is           |
|            |       | continuously tracked. The Risk Registry and    |
|            |       | the risk mitigation forms are maintained in    |
|            |       | the Document data base, organized by WBS L2    |
|            |       | area. Only the production of fully tested CCDs |
|            |       | and the fabrication of the lenses are          |
|            |       | classified as high risk. These risks have been |
|            |       | substantially mitigated by the R&D program for |
|            |       | the CCDs, and initiation of early procurement  |
|            |       | of the lenses, although they are still         |
|            |       | considered high risk.                          |
|            |       |                                                |
|            |       | Overall project risk has been mitigated by     |
|            |       | choosing technologies and solutions that have  |
|            |       | already been demonstrated or only require      |
|            |       | modest development to meet the DECam           |
|            |       | requirements.                                  |
|            |       |                                                |
|            |       | **Committee Response**: Committee acknowledges |
|            |       | that a satisfactory risk management plan has   |
|            |       | been written. Aggressive risk and issue        |
|            |       | management must be instituted within the       |
|            |       | day-to-day management of the project and fully |
|            |       | integrated into the schedule updates and       |
|            |       | status, cost and contingency analysis.         |
|            |       | Likewise global and cross cutting risks are    |
|            |       | not identified within the risk register. The   |
|            |       | risk register documents risks but needs to be  |
|            |       | adopted as an active management tool and be    |
|            |       | fully integrated and implemented for all       |
|            |       | activities, especially at the systems level.   |
+------------+-------+------------------------------------------------+
| 6\.        | Sa    | **Project Response**: The funding profile is   |
| Funding    | tisfa | consistent with the resource loaded schedule.  |
| Profile    | ctory |                                                |
|            |       | **Committee Response**: Committee concurs.     |
+------------+-------+------------------------------------------------+
| 7\.        | Sa    | **Project Response**: The project control and  |
| Project    | tisfa | EVMS systems are developed in compliance with  |
| Con        | ctory | the Fermilab policy and using standard         |
| trols/EVMS |       | Fermilab practices and tools. Earned value     |
|            |       | tracking, and variance reports are in use.     |
|            |       | Monthly reports have been prepared starting    |
|            |       | Aug. 2007. These contain the Cost Performance  |
|            |       | Reports which indicate earned value. The       |
|            |       | Fermilab Director and the DOE Site office held |
|            |       | a joint review of the EVMS system in use for   |
|            |       | DECam. The overall conclusion of the review    |
|            |       | was "The DECam project has satisfactorily      |
|            |       | implemented the bulk of the elements required  |
|            |       | for a 'self-certified' EVMS".                  |
|            |       |                                                |
|            |       | **Committee Response**: Committee concurs.     |
+------------+-------+------------------------------------------------+
| 8\. Basis  | Sa    | **Project Response**: The DECam project        |
| of Design  | tisfa | maintains a Technical Requirements and         |
|            | ctory | Specifications document which provides the     |
|            |       | design basis for all of the DECam subsystems   |
|            |       | and DECam as a whole. The DECam design is well |
|            |       | developed and this is documented in the        |
|            |       | Technical Design Report. Several areas of work |
|            |       | which are based on previous R&D are ready for  |
|            |       | production (CCD Processing and Packaging).     |
|            |       | Several Make vs Buy comparisons have resulted  |
|            |       | in decisions to buy complete systems rather    |
|            |       | than devote engineering resources to their     |
|            |       | design and fabrication. System requirements    |
|            |       | have been identified, and are consistent with  |
|            |       | the specifications and preliminary drawings.   |
|            |       | The major mechanical systems have individual   |
|            |       | Requirements and Specifications Documents      |
|            |       | which are actively maintained and used to      |
|            |       | communicate design details between subsystems, |
|            |       | or for procurement of complete systems from a  |
|            |       | vendor. The major software systems have        |
|            |       | Interface Control Documents.                   |
|            |       |                                                |
|            |       | **Committee Response**: Committee concurs.     |
+------------+-------+------------------------------------------------+
| 9\. Design | Sa    | **Project Response**: Since receiving CD-0     |
| Review     | tisfa | approval, the DECam Project as a whole has     |
|            | ctory | been reviewed 5 times:                         |
|            |       |                                                |
|            |       | > 1\) Fermilab Director's CD-1 Review July     |
|            |       | > 2006                                         |
|            |       | >                                              |
|            |       | > 2\) Joint DOE/NSF review of DES and CD-1     |
|            |       | > Review of the DECam Project in May 2007      |
|            |       |                                                |
|            |       | 3\) Preliminary CD2/3a Director's Review of    |
|            |       | DECam Oct. 2007.                               |
|            |       |                                                |
|            |       | 4\) CD2/3a Directors' Review of DES Dec.       |
|            |       | 11-13, 2007                                    |
|            |       |                                                |
|            |       | 5\) Fermilab and DOE EVMS review of DECam Dec. |
|            |       | 18-19 2007                                     |
|            |       |                                                |
|            |       | The review reports and the project responses   |
|            |       | are part of review materials for each          |
|            |       | subsequent review. In addition, 5 reviews of   |
|            |       | subsystem designs were convened by the DECam   |
|            |       | Project manager in the past year and           |
|            |       | additional reviews are scheduled throughout    |
|            |       | the project. These reviews are a necessary     |
|            |       | step before initiation of major procurements   |
|            |       | and design cycles. Safety reviews for systems  |
|            |       | commissioned as part of CCD R&D have been      |
|            |       | conducted by the appropriate ES&H committees   |
|            |       | in PPD.                                        |
|            |       |                                                |
|            |       | **Committee Response**: Committee concurs.     |
+------------+-------+------------------------------------------------+

+------------+-------+------------------------------------------------+
| **EIR      | **SC  | **Comment**                                    |
| Element**  | R     |                                                |
|            | eview |                                                |
|            | Team  |                                                |
|            | As    |                                                |
|            | sessm |                                                |
|            | ent** |                                                |
+============+=======+================================================+
| 10\.       | Sa    | **Project Response**: System functions and     |
| System     | tisfa | requirements flow down from the DES Science    |
| Functions  | ctory | objectives and are captured in the DECam       |
| and        | with  | Technical Requirements and Specification       |
| Re         | co    | Document. This document is prepared and        |
| quirements | mment | maintained by the DECam Project scientist. The |
|            |       | requirements and specifications are consistent |
|            |       | with the key performance parameters and the    |
|            |       | deliverables described in the Project          |
|            |       | Execution Plan and with the mission need. The  |
|            |       | DECam Technical Requirements and Specification |
|            |       | Document will be used to define the detailed   |
|            |       | technical tests that ensure that the delivered |
|            |       | DECam meets the requirements. Each major       |
|            |       | mechanical system has a specifications and     |
|            |       | design document which include references to    |
|            |       | drawings that serve as the mechanical          |
|            |       | interface control. The major software systems  |
|            |       | have functional requirements documents and     |
|            |       | interface control documents. The design        |
|            |       | requirements are reflected in the baseline,    |
|            |       | including safety aspects.                      |
|            |       |                                                |
|            |       | The Mission Need for the project is contained  |
|            |       | in the CD-0 Approval for Ground-Based Dark     |
|            |       | Energy Experiment that was approved on         |
|            |       | November 22, 2005. To lay out a roadmap for    |
|            |       | studying the nature of dark energy, the High   |
|            |       | Energy Physics Advisory Panel (HEPAP) and      |
|            |       | Astronomy and Astrophysics Advisory Committee  |
|            |       | (AAAC) commissioned a subpanel, the Dark       |
|            |       | Energy Task Force (DETF).                      |
|            |       |                                                |
|            |       | In their spring 2006 report, the DETF          |
|            |       | recommended a mix of experiments with          |
|            |       | independent and complementary measurements to  |
|            |       | address the nature of dark energy. The report  |
|            |       | defined a four-stage program to understand the |
|            |       | nature of dark energy, with Stage III being    |
|            |       | mid-term projects and Stage IV being longer    |
|            |       | term, next generation, projects. They defined  |
|            |       | a figure of merit (FOM) based on the errors in |
|            |       | measurements of dark energy quantities. Using  |
|            |       | this FOM, they concluded that, given           |
|            |       | uncertainties in projected systematic errors,  |
|            |       | no single method of measuring dark energy      |
|            |       | could robustly constrain both parameters.      |
|            |       | Rather, they concluded that tight constraints  |
|            |       | would need to come from multiple dark energy   |
|            |       | methods.                                       |
|            |       |                                                |
|            |       | DES will measure the dark energy parameters    |
|            |       | with the four dark energy methods recommended  |
|            |       | by the DETF and will provide improvement in    |
|            |       | the FOM between a factor of 4 and 5 over       |
|            |       | current (Stage II) projects. Thus the DES      |
|            |       | satisfies the DETF recommendations for a       |
|            |       | mid-term (Stage III) project.                  |
|            |       |                                                |
|            |       | In a letter dated June 23, 2006, HEPAP's P5    |
|            |       | prioritization subpanel recommended that DOE   |
|            |       | and NSF jointly pursue the DES project, which  |
|            |       | can provide significant advances in our        |
|            |       | knowledge of dark energy.                      |
|            |       |                                                |
|            |       | **Committee Response**: Committee concurs.     |
|            |       | Configuration management should be             |
|            |       | strengthened within the DECam project as error |
|            |       | budget, requirement and specification flow     |
|            |       | downs and traceability are only nascent or     |
|            |       | missing.                                       |
+------------+-------+------------------------------------------------+
| 11\.       | Sa    | **Project Response**: The DECam project does   |
| Sust       | tisfa | not involve building any new facilities at     |
| ainability | ctory | Fermilab or elsewhere. The camera will be      |
|            |       | installed on a 4-m telescope in Chile which is |
|            |       | operated by NOAO (CTIO) and funded by NSF.     |
|            |       |                                                |
|            |       | **Committee Response**: Committee concurs.     |
+------------+-------+------------------------------------------------+
| 12\.       | Sa    | **Project Response**: A Preliminary Hazard     |
| Hazards    | tisfa | Assessment was made as part of the CD-1        |
| Analysis   | ctory | preparations, and subsequently updated. Based  |
|            |       | on this assessment, a Preliminary Safety       |
|            |       | Assessment Document has been written to        |
|            |       | describe the mitigations of these hazards. The |
|            |       | PHA has been signed and forms part of the      |
|            |       | approved CD-1 documentation. The PSAD has been |
|            |       | reviewed by the PPD ES&H liaison.              |
|            |       |                                                |
|            |       | **Committee Response**: Committee concurs.     |
+------------+-------+------------------------------------------------+
| 13\. Value | Sa    | **Project Response**: Value                    |
| Ma         | tisfa | Management/Engineering is fundamental to the   |
| nagement/E | ctory | execution of the project and is described in   |
| ngineering |       | our Technical Design Report. The value         |
|            |       | engineering processes that led to the current  |
|            |       | design choices for the CCDs, the Front End     |
|            |       | Electronics and optical design are described.  |
|            |       | In addition, detailed value engineering        |
|            |       | documents describe the optimization of the CCD |
|            |       | thickness and the placement of electronic      |
|            |       | components in the readout chain. The design of |
|            |       | the cooling system has undergone detailed      |
|            |       | technical reviews as part of the value         |
|            |       | engineering process and finalization of the    |
|            |       | design.                                        |
|            |       |                                                |
|            |       | **Committee Response**: Committee concurs.     |
+------------+-------+------------------------------------------------+
| 14\.       | Sa    | **Project Response**: The DECam Project        |
| Start-up   | tisfa | Scientist is responsible for developing the    |
| Test Plan  | ctory | testing plans which will be performed prior to |
|            | with  | shipping to Chile and again prior to           |
|            | co    | installation on the telescope. It is linked to |
|            | mment | the CD-4 deliverables identified in the PEP.   |
|            |       | It identifies the acceptance and operational   |
|            |       | system tests required to demonstrate design    |
|            |       | performance and safety requirements. Cost and  |
|            |       | schedule durations for these tests are         |
|            |       | appropriately incorporated into the baseline.  |
|            |       | A testing plan has been drafted and is now     |
|            |       | under document control in the document         |
|            |       | database.                                      |
|            |       |                                                |
|            |       | **Committee Response**: Committee concurs. The |
|            |       | present test plan is only a framework draft    |
|            |       | and will require significant development in    |
|            |       | successive revisions before it is a            |
|            |       | comprehensive and complete document.           |
+------------+-------+------------------------------------------------+
| 15\.       | Sa    | **Project Response**: The Project Execution    |
| Project    | tisfa | Plan reflects and supports the way the project |
| Execution  | ctory | is being managed. It has been updated from the |
| Plan       |       | CD-1 Preliminary PEP with the new funding      |
|            |       | profile and the resulting schedule. It is      |
|            |       | consistent with other project documents and    |
|            |       | establishes a plan for successful execution of |
|            |       | the project.                                   |
|            |       |                                                |
|            |       | **Committee Response**: Committee concurs.     |
+------------+-------+------------------------------------------------+
| 16\.       | Sa    | **Project Response**: The Acquisition Strategy |
| A          | tisfa | is consistent with the way the project is      |
| cquisition | ctory | being executed. No significant changes in      |
| Strategy   |       | acquisition approach have occurred since its   |
|            |       | signature by the AE for CD-1 in October 2007.  |
|            |       | It still represents the best value to the      |
|            |       | government.                                    |
|            |       |                                                |
|            |       | **Committee Response**: Committee concurs.     |
+------------+-------+------------------------------------------------+
| 17\.       | Sa    | **Project Response**: The IPT has been formed  |
| Integrated | tisfa | and meets monthly or as needed. It is chaired  |
| Project    | ctory | by the Federal Project Director. The charter   |
| Team       | with  | as well as the full membership is incorporated |
|            | co    | into the PEP.                                  |
|            | mment |                                                |
|            |       | **Committee Response**: Committee concurs. The |
|            |       | IPT presently lists the Fermi Site Office      |
|            |       | Manager as a member of the IPT. This is not    |
|            |       | necessary as the Federal Project Director      |
|            |       | represents the FSO Manager.                    |
+------------+-------+------------------------------------------------+