![](media/image1.wmf){width="2.1222222222222222in"
height="0.7326388888888888in"}

Final Report

Director's

CD-2/3a Review

of

the NOvA Project

June 4-6, 2007

This page intentionally left blank

Table of Contents

[Executive Summary 5](#__RefHeading___Toc169312687)

[1.0 Introduction 7](#__RefHeading___Toc169312688)

[2.0 Science 8](#__RefHeading___Toc169312689)

[3.0 Site and Building (WBS 1/2.1) 11](#__RefHeading___Toc169312690)

[4.0 Commodities -- Scintillator/Fiber/PVC (WBS 1/2.2, 1/2.3 & 1/2.4)
14](#__RefHeading___Toc169312691)

[5.0 Extrusion Module Production (WBS 1/2.5)
16](#__RefHeading___Toc169312692)

[6.0 Electronics, Trigger DAQ (WBS 1/2.6 & 1/2.7)
20](#__RefHeading___Toc169312693)

[7.0 Far and Near Detector Assembly (WBS 1/2.8 & 2.9)
22](#__RefHeading___Toc169312694)

[8.0 Accelerator Upgrades (WBS 1/2.0.1 & 1/2.0.2)
32](#__RefHeading___Toc169312695)

[9.0 NuMI Beamline Upgrades (WBS 1/2.0)
38](#__RefHeading___Toc169312696)

[9.1 Beamline/TargetModifications 38](#__RefHeading___Toc169312697)

[9.2 Shielding 39](#__RefHeading___Toc169312698)

[10.0 Cost 40](#__RefHeading___Toc169312699)

[11.0 Schedule 44](#__RefHeading___Toc169312700)

[12.0 Project Management (WBS 1.9 & 2.10)
48](#__RefHeading___Toc169312701)

[13.0 Charge Questions 51](#__RefHeading___Toc169312702)

[Appendices 54](#__RefHeading___Toc169312703)

[Cost Estimates 55](#__RefHeading___Toc169312704)

[Charge 57](#__RefHeading___Toc169312705)

[Agenda 61](#__RefHeading___Toc169312706)

[Report Outline and Reviewer Writing Assignments
63](#__RefHeading___Toc169312707)

[Reviewer Assignments for Breakout Sessions
65](#__RefHeading___Toc169312708)

[Reviewers' Contact Information 66](#__RefHeading___Toc169312709)

[Participant List 68](#__RefHeading___Toc169312710)

[Table of Recommendations 71](#__RefHeading___Toc169312711)

This page intentionally left blank

[]{#__RefHeading___Toc169312687 .anchor}Executive Summary

Technical

The NOvA collaboration has prepared a draft Technical Design Report
(TDR), estimated cost, and schedule for an 18 kton neutrino detector to
be located in Northern Minnesota that will detect the appearance of
electron neutrinos in a beam from an upgraded NuMI muon neutrino source
at a power level of \~700 kW at Fermilab. The draft TDR is a quite
comprehensive document describing an \~30% design of the NOvA detector
and the proposed accelerator and NuMI upgrades (ANU).

A well-advanced design for the Far Detector Building has been prepared.
The DOE is entering into a Cooperative Agreement with the University of
Minnesota (UMN) to finish the design and build the building (and operate
the facility once it is complete and perform HEP Research).

A NOvA R&D program is underway to address key technical questions prior
to placing major component procurement contracts.

Cost

The initial roll up in Cobra of the Resource Loaded Schedule (RLS) from
Open Plan for the detailed cost estimate of NOvA results in a total
project cost (TPC) of \$297.4M including a contingency of 31% overall.
The present cost estimate for the 18 kton scope seems generally to have
quite a sound basis, but needs to be scrubbed carefully by the NOvA team
to identify errors, duplications, and other problems.

The Site and Building cost increased by 24% (to \$57.M) from CD-1 (with
a 25 kton detector). The increases were due to increased HVAC, crane,
and electrical utility capacity (all driven by technical requirements,
eg the detector plane bonding glue) and University of Minnesota (UMN)
Project Management support.

Modest increases in the detector cost were due to Electronics
Production, Near Detector Assembly, and Project Management. There was a
significant decrease in the estimate for PVC Extrusions.

The ANU costs have increased \~10% since the Super NuMI (SNuMI)
Director's Review.

Schedule

The NOvA schedule shows initial data taking beginning with the 1^st^
super block in September 2011 and an April 2013 completion. The \~6000
line construction schedule in Open Plan has been resource loaded with
logic included and yields a critical path that includes the Far Detector
Building, block pivoter, and detector module assembly. The ANU upgrades
will be finished in time to meet the NOvA scheduled beam requirements.

The far detector assembly schedule (the long pole in the tent) extending
over four years is thought to be reasonable. A 100% contingency on labor
for this task is included to allow double shifting if needed to maintain
the schedule. The schedule shows detector site and building final design
beginning late this fiscal year, road work at the beginning of Spring
2008, and excavation and concrete work beginning in the Summer 2008. The
CD-3a request also includes long-lead time detector items and specific
ANU component procurements.

Management

There has been a \$260M cap placed on NOvA by the Office of Management
and Budget (OMB). So, the detector size will need to be scaled down to
fit under this cap. The "scrubbing" of the 18 kton cost mentioned above
and the descoping will need to be completed prior to a DOE Lehman
CD-2/3a Review. This review is presently scheduled for July 17-19 (date
was moved to August 11-13 shortly after completing the Director's
review).

A reasonably well staffed (for this stage of the project) NOvA project
team is in place and responsible for accomplishing the TDR and this
initial schedule and cost estimate. They must now accomplish the
scrubbing and descoping to achieve a self-consistent scope, detailed
schedule, and sound cost estimate for a \$260M NOvA. Rough scaling
calculations indicate this would result in about a 14 kton detector
mass.

The Acquisition Strategy has been approved by DOE, but many of the
"management documents" needed for DOE projects at the CD-2/3a stage have
been prepared in draft form, including:

-   Project Execution Plan

-   Project Management Plan

-   Security Vulnerability Assessment Report

-   Procurement Plan for NOvA Cost Drivers

-   Risk Management Plan

-   Integrated Safety Management Program

-   Configuration Management Program

-   Quality Assurance Program

-   Environmental Assessment and

-   Environmental Assessment Worksheet.

Some additional work and significant effort will be required to bring
this suite of documents current with the to-be-developed \$260M NOvA
scope, cost, and schedule. Finally, a [polished]{.underline} (EIR ready)
full set of information for the DOE Lehman CD-2/3a Review will need to
be provided \~ 1 week prior to that review on an easily accessible and
user-friendly webpage.

[]{#__RefHeading___Toc169312688 .anchor}1.0 Introduction

A Director's CD-2/3a Review of the NOvA Project was held on June 4-6,
2007. The charge included a list of topics to be addressed as part of
the review. The assessment of the Review Committee is documented in the
body of this report.

Each section in the report is generally organized by Findings, Comments
and Recommendations. Findings are statements of fact that summarize
noteworthy information presented during the review. The Comments are
judgment statements about the facts presented during the review and are
based on reviewers' experience and expertise. The comments are to be
evaluated by the project team and actions taken as deemed appropriate.
Recommendations are statements of actions that should be addressed by
the project team. Progress on the recommendations is to be reported on
during future NOvA Working Group Meetings (WGMs). A response to
recommendation(s) is expected and actions taken will be reported on
during future reviews.

Reference materials for this review are contained in the Appendices.
Appendix A is NOvA's project cost estimate/contingency spreadsheet and
the Review Committee's assessment of the cost estimate/contingency
spreadsheet documented in the notes. The Charge for this review is shown
in Appendix B. The review was conducted per the agenda shown in Appendix
C. The Reviewer's assignments are noted in Appendix D and E, and their
contact information is listed in Appendix F. The Review Participants are
listed in Appendix G. Appendix H is a table that contains all the
recommendations included in the body of this report.

[]{#__RefHeading___Toc169312689 .anchor}2.0 Science

**Summary**

-   The proposed detector technology has outstanding background
    rejection and \> 30% detection efficiency for ν~e~ appearance.

-   The Near and Far detector configurations are very well optimized to
    the physics goals.

-   Concern: The physics sensitivity scales with
    sqrt(mass\*efficiency\*time\*pot)

-   Descoping from 25 kT to 18 kT reduces the physics sensitivity by
    22%\*, reduction to 14 kT reduces the sensitivity by 39%.

(\* this factor also includes a reduction in the protons-on-target
assumption from 6.5 to 6.0 E20/year.)

Findings

The NOvA experiment is intended to:

-   Detect muon-electron neutrino oscillations with sensitivity more
    than an order of magnitude greater than the present experiments. In
    particular, to measure the mixing angle sin\^2 (2 theta13) with a
    sensitivity of 3 sigma at sin\^2 (2 theta13) \> 0.01.

-   To use the differential effects of passage through matter on
    neutrinos and anti-neutrinos at long oscillation lengths to
    determine the mass hierarchy for neutrino species.

-   To perform precision measurements of the \'atmospheric\' neutrino
    mixing by detecting the disappearance of muon neutrinos at a far
    detector.

    To do this requires:

-   A high intensity neutrino and anti-neutrino beam, with beam energy
    spread comparable to the energy modulation expected in neutrino
    oscillations.

-   A high mass neutrino detector located at

```{=html}
<!-- -->
```
-   the first maximum in the sin\^2 (Delta M_23\^2 L/E) oscillation peak
    for \'atmospheric\' neutrinos

-   with as much matter between the beam origin and the detector as
    possible to maximize the matter effects.

```{=html}
<!-- -->
```
-   The ability to distinguish muon-\>electron neutrino interactions
    from

```{=html}
<!-- -->
```
-   fake electron neutrino signatures from neutral current events and
    cosmic ray interactions and

-   contamination by electron neutrinos in the muon beam. This can only
    be done by understanding the beam spectrum itself as the final state
    signature is identical to the oscillation signal.

```{=html}
<!-- -->
```
-   The collaboration have designed and optimized a detector/system to
    perform this measurement using the high intensity NUMI beamline
    currently running at Fermilab. It will be upgraded to 700 kW by the
    time NOvA begins data taking. The NOvA collaboration will locate
    their 18 kT liquid scintillator far detector 12 km off of the NUMI
    beamline axis 810 km from Fermilab in order to achieve a maximally
    monochromatic neutrino beam at the first oscillation maximum. A 218
    T near detector will be located at an angle of 14 mRadians off of
    the NUMI beamline at Fermilab.

Comments

-   Detector

```{=html}
<!-- -->
```
-   Neutrino detectors with sufficient segmentation and little dead
    material can distinguish pi0 production in neutral current (NC) muon
    neutrino interactions from Charged Current (CC) electron neutrino
    interactions with good resolution. The collaboration has chosen to
    optimize their detector for rejection of NC backgrounds by making it
    \~70% active and using liquid scintillator as the active medium.
    Backgrounds from cosmic ray photons are reduced by a concrete and
    barite overburden and by the short NUMI beam pulse. The
    reconstruction efficiency for nu_e within the fiducial region is \~
    31% with background rejection of 3:1000.

-   The energy resolution of the detector needs to be small enough that
    detector resolution does not significantly broaden the measured beam
    energy beyond the 25% beam spread. For electron neutrino CC events,
    the baseline design achieves 6% resolution for all events and 4%
    resolution for quasi-elastic interactions. For muon neutrino events
    the event energy resolution requirement for precision measurements
    of the atmospheric neutrino parameters is 4%. The resolutions for
    both nu_e and nu_mu interactions are well within the design
    criteria.

-   Understanding of electron neutrino contamination in the beam itself
    requires a 20 T fiducial volume near detector, which samples the
    beam before oscillation. The collaboration has decided on a fixed
    218 T near detector located in a new enclosure off of the NUMI
    access tunnel.

-   Optimizations of the detector design and reconstruction algorithms
    have been done using full simulations of the beam and the detector
    response.

**General conclusion** - the detector performance and location
parameters have been well optimized to maximize the physics impact of
the experiment. The remaining parameter is the product (mass x
efficiency x running time x neutrino flux) which is largely determined
by the resources available.

-   Fiducial mass

```{=html}
<!-- -->
```
-   The far detector configuration presented for this review has been
    reduced from 25kT mass at the time of the 2006 CD1 review to 18 kT
    mass using the same detector technology. The beam flux assumed is
    now 6.0 10\^20 pot/year for 6 years, while some earlier proposals
    assumed 6.5 10\^20 pot/year.

-   The collaboration have defined a figure of merit S/Sqrt(B) where S
    is signal for sin2 2theta_13 at the Chooz limit of 0.1 and B is the
    background from beam contamination and other interactions
    misidentified as electron neutrino interactions. A figure of merit
    of 30 corresponds to a 3 sigma sensitivity to sin\^2 2theta_13 \>
    0.01.

-   Assuming an equivalent six year running time the change in beam flux
    assumption and fiducial mass lowers the optimal FOM from 33 for a 25
    kT detector, 6.5 10\^20pot/year and a 6 year run to 27 for an 18kT
    detector and 6.0 10\^20 pot/year. We were not certain if the
    improvements in algorithmic efficiency and background rejection were
    included in the numbers given in the TDR so these may be
    underestimates.

-   We heard that a further reduction in mass to 14kT may be necessary
    to stay within the cost envelope. This would reduce the FOM to below
    25 for a 6 year run.

-   However, if one states this in terms of sensitivity to sin\^2
    2theta_13, the sensitivity to the physics observable for a 6 year
    run changes from 0.0094 at 25 kT to 0.012 at 18 kT to 0.014 at 14
    kT.

-   Addressing decreases in mass by increasing the running time implies
    a substantially longer run which may not be feasible.

```{=html}
<!-- -->
```
-   **Observation** - sections 4 and section 6 of the TDR are currently
    inconsistent in their estimates of the sensitivity. Section 6 needs
    to be updated to reflect the new baseline pot/year and mass numbers
    as well as the improvements in algorithms.

Recommendations

1.  The mass and flux estimates in the TDR need to be made consistent.

2.  **The continued shrinkage of the NOvA far detector is an area of
    concern**. The collaboration needs to come up with a consistent plan
    which allows the project to reconcile the cost guidance with maximal
    physics sensitivity.

[]{#__RefHeading___Toc169312690 .anchor}3.0 Site and Building (WBS
1/2.1)

Findings

-   The site preparation and access road drawings for the Ash River
    Trail site have been prepared by Burns and McDonnell and are 95%
    complete. The remaining 5% design effort includes revising the
    excavation plan for the building to match the current plans for the
    detector hall concrete design. The review committee was provided
    with review comments and resulting responses and actions from
    comment and compliance and QA reviews for the site access road
    design. Several comments contained within these reviews were from
    the University of Minnesota (UM) stakeholders, which confirms that
    the future site owners have been involved in this design effort.

-   The Ash River site building drawings are between 30% and 55%
    complete and have been prepared in FESS Engineering. The design team
    plans to issue these design drawings for a comment and compliance
    review of all stakeholders in the near future. NOvA plans to
    complete the structural concrete drawings within FESS Engineering
    and couple them with the site preparation and access road scope to
    ensure the excavation and structural concrete work is fully
    coordinated.

-   CD-3a approval will be sought for the site preparation and access
    road scope along with structural concrete scope. These packages are
    planned to bid concurrently in the first quarter of FY 2008. This
    methodology is being implemented to support issuing the contractor
    Notice to Proceed in the third quarter of FY 2008.

-   A cooperative agreement is being developed between DOE and U Minn to
    support construction of the building by U Minn. This will clarify
    many open logistics questions regarding final design and
    construction.

-   The technical requirements for the facility have been modified since
    the CD 1 review. These changes have resulted in substantial
    increased cost for the facility and its systems.

-   Extensive risk assessment documentation has been completed for most
    of the major elements of construction.

-   The team has three independent cost estimates completed and is
    prepared to perform reconciliation between them. This is a
    conscientious action being taken to incorporate the best available
    information into the project estimate.

-   The team has performed Monte Carlo analysis to assess adequacy of
    contingency.

-   A review of the Basis of Estimate document revealed many actual
    quotes and bids for proposed work. This information is dependable to
    support the pricing proposals and assigned contingencies included in
    the project.

-   The project has developed an excellent option for wetland banking.
    This appears to be a low-cost opportunity to manage the mitigation
    of wetlands needed by the project.

-   NOvA intends to construct a space to contain the near detector in
    the MINOS cavern downstream of the MINOS access shaft. A preliminary
    design exists and the design team is prepared to proceed with
    selecting a consultant to further develop the concept to obtain a
    more accurate construction estimate.

Comments

-   The Cooperative Agreement between DOE and U Minn is not yet
    complete. This document is needed to allow further development of
    MOUs and responsibility matrixes.

-   The Environmental Assessment process is not complete. The EA has
    been prepared to include the work for the Ash River site and the
    work planned at Fermi Lab. The EA will require public review by
    Illinois, Wisconsin and Minnesota. This document should be finished
    and sent out for comment as soon as possible. It is our assumption
    that the decision on the Environmental Assessment must be complete
    prior to initiation of work on either site.

-   The site preparation and access road plans do appear to be
    substantially complete. Coordination of the building excavation and
    structural concrete is key to keeping the project on schedule and
    minimize change orders so the reviewers concur with the approach to
    complete the structural concrete drawings and include that scope in
    the initial procurements done under the UM CA. We suggest the site
    borings and rock coring data be shown on the site preparation and
    access road drawings for completeness.

-   Activities required to support the requirements of CD-3a include
    successful completion of the Lehman review in July, followed by an
    External Independent Review (EIR date to be determined), and finally
    the ESAAB approval of CD 2/3a in the fourth quarter of FY 2007.
    These activities are not currently shown in the schedule.

-   A drill down on the Open Plan schedule revealed inconsistencies
    between Open Plan and the Basis of Estimate data book (specifically
    WBS 2.1.2.3.5.1.1, 2.2.2.3.5.1.2, 2.1.2.3.5.8.3, 2.1.2.3.5.8.5).
    These documents should be reviewed and reconciled for consistency.

-   The review team feels NOvA should retain the services of a
    consulting firm in the near future to continue the development of
    the near detector excavation in the MINOS Hall.

Recommendations

3.  It is recommended that the project develop a responsibility matrix
    that exhibits the various requirements and responsibilities of the
    individual team members related to the construction and oversight of
    the road, site and facility.

4.  It is important to further develop the schedule incorporating the
    activities leading to an approval of CD-3a.

[]{#__RefHeading___Toc169312691 .anchor}4.0 Commodities --
Scintillator/Fiber/PVC (WBS 1/2.2, 1/2.3 & 1/2.4)

Findings

-   Commodity items are estimated to account for approximately 40% of
    the Total Estimated Cost (TEC).

-   For all three commodities, actual solicitations were issued, and
    fixed pricing (indexed and subject to various escalators identified)
    proposed by suppliers was used as the basis of estimates. A more
    recent quote on mineral oil pricing is available but is not
    currently used in the cost and contingency estimate.

-   The scintillator is made by combining purchased components, the
    largest being the mineral oil. A change since the last review is
    that the mixing is done by a commercial vendor, instead of in-house.
    Most items can be competitively bid, except for the wave shifters,
    for which only one viable source has been identified.

-   At this time there is only a sole supplier identified for the
    wave-length shifting fiber. Pricing is known to be not strongly
    dependent on the concentration of the dye.

-   Since the last review, the 16-cell extrusion was chosen as the
    baseline for forming the PVC extrusions due to the high cost of
    manufacturing a 32-cell die and resin flow problems encountered
    during R&D for the 16-cell prototype. The costs of the production
    dies are included in the contract with the extruder.

Comments

-   The terms "Purchase" and "Procure" are used in many different ways
    throughout the project planning documents, representing different
    segments of the procurement process

-   Continuing to request estimates from the same suppliers for
    dwindling quantities could have a negative affect on estimates and
    final pricing proposals.

-   Extensive QA/QC procedures were presented for each commodity at
    appropriate stages of the acquisition process. Onsite project QA
    reps will be located at production sites when appropriate.

-   Shipping and handling costs for the extrusions seem to be well
    developed.

-   The Level 2 managers did not have time to review and correct the
    Open Plan documents before the Review.

Recommendations

5.  Scrub the costs and schedules in Open Plan.

6.  Develop a standard procurement milestone plan to use across the
    commodities.

7.  Use the current pricing estimate available for the mineral oil and
    recalculate the contingency.

[]{#__RefHeading___Toc169312692 .anchor}5.0 Extrusion Module Production
(WBS 1/2.5)

Findings

-   The module construction team presented the latest status on the
    design, testing and performance of the PVC modules. The
    presentations covered the module design, factory machines, tooling,
    module production and the photodetector interface. Currently the
    production concept has work performed at two factories, one located
    at Fermilab and one located in Minnesota.

-   The module consists of 32 cell PVC extrusion assemblies made from
    two 16 cell extrusions that are glued together, a bottom seal
    assembly, a top fiber manifold and seal assembly, WLS fibers and the
    optical connector/interface to the readout APD. There are two
    different types of modules: Vertical and Horizontal. They differ in
    the mechanical structure of the extrusions: the vertical have
    thicker walls for strength. So far only horizontal extrusions have
    been fabricated at full scale.

-   The raw 16 cell extrusions are first delivered to the Fermilab
    factory where they are unpacked, entered into the database (via
    bar-code ID), visually inspected and measured (thickness, width, and
    length) and sorted according to the measurements. All measurements
    are entered into the database. A 32 cell module is then made by
    gluing two 16 cell extrusions together. The 32 cell module is then
    cut to length and abraded near the ends to facilitate gluing in
    subsequent assembly steps. At this time the cutting is done by hand
    using a circular saw with a carbide tip and an edge guide mounted on
    the module. The completed modules are then stacked in groups of 30
    wrapped and shipped to the Minnesota Factory. Designs for pallet
    movers (commercial air) lifting fixtures, rolling tables and gluing
    tables are complete and have, for the most part, been tested (not
    yet for full length for gluing table).

-   The 32 cell extrusions modules are delivered to the Minnesota
    factory by truck on pallets of 30. The Minnesota factory will be
    located off-campus at a rented facility of approximately 60k ft^2^.
    At the Minnesota facility, the 32 channel extrusion assemblies are
    first stacked, interleaved with 3" spacers, in preparation for all
    operations. The top fiber manifold assembly is then put on the 32
    channel extrusion. This assembly includes the WLS fibers (strung in
    each of the 32 cells), side and center seals, raceways, fiber cover,
    snout and the optical connector and the manifold cover. The WLS
    fibers are routed to the optical connector during this step. There
    are various glue steps in this operation with the resultant assembly
    being liquid tight. The optical connector gets its diamond polish
    (fly-cut) at this time. Liquid scintillator fill, vent and drain
    ports are in this assembly. After the top fiber manifold assembly is
    complete and all glues cured, the end seal is glued to the other end
    of the extrusion cells. Two glue systems are used in both the end
    seal and fiber manifold. 3M 2216, a 2-component epoxy, is used to
    form the barrier seal between the scintillator and the outside
    world. It is inert and does not affect the scintillator properties
    nor are its properties affected by the scintillator. The second
    adhesive is Devcon Plastic Welder. It is structurally strong and
    provides the main strength to the bonding system. It is not in
    contact with the scintillator. If it comes in contact with the
    scintillator it will affect the scintillator performance. Its
    properties also appear to be altered when it gets in contact with
    the scintillator. The barrier glue, 2216, prevents this from
    happening. After cure the completed module is tested for leak
    tightness (using pressurized gas and a bubble detector) and fiber
    continuity. The completed modules passing test are then packed and
    made ready for shipment to the NOvA site. The current concept is
    that modules that fail testing are not repaired. A very low failure
    rate is required and is the current expectation.

-   The PVC module team has evaluated the health and safety aspects of
    factory operations and presented a good plan to assure the health
    and safety of all personnel involved with the production. They have
    paid particular attention to lifting fixtures and are working on a
    detailed training program for all personnel. Issues due to glue
    fumes will be dealt with by providing appropriate ventilation.

-   Major productions risks have been identified, but we have not seen a
    detailed risk register for this WBS project element.

-   Since the CD-1 review, the estimated cost for the PVC modules has
    increased from \$13.8M to \$21.1M. The cost increase is entirely due
    to labor increases. \[Note, the CDR-1 review committee commented
    that they thought the labor estimates at that time were thin and
    asked the team to re-evaluate their labor estimates.\]

Comments

-   The committee commends the PVC module team on the excellent progress
    that they have made since the CD-1 review. Most factory machine and
    tooling designs are complete and procedures are well developed.
    Concerns regarding the End Plate and Fiber Manifold gluing/seals
    have been addressed. Regarding the End Plate seals, we note that the
    team has now performed 57 trial seals on 16 cell extrusions. The
    procedures have now stabilized and the last 10 trials sealed
    successfully. Obviously this is still a small sample and must be
    monitored carefully during production for the prototype in order to
    guarantee that there are no subtle problems occurring at a small
    level. The team should make sure that all operations are controlled
    as tightly as possible so that production errors are minimized.
    Automation should be implemented where ever possible.

-   The interference between vertical and horizontal fiber manifolds in
    some regions of the detector is still a concern with the clearance
    in these areas only being a few mm. This clearance seems tight to
    the committee.

-   We feel that the space in the Wide Band Hall is far from optimal for
    NOvA operations. Given the nature of the work required for the PVC
    extrusion processing, the current layout for Factory 1 (WBH)
    requires a great deal of otherwise unnecessary handling of the
    extrusions, including a move of 20 vertical ft. from the loading
    dock to the pit and back up again. The estimate given by the NOvA
    team was that approximately \$1M in labor could be saved if an
    optimized layout for Factory 1 were available. Local leasing costs
    for industrial space are low enough that a local offsite factory
    staffed by Fermilab personnel may be cost effective.

-   It was noted that construction for the Minerva Detector does not end
    until 9/30/2009 according to Minerva's current project plan. This
    only leaves 1 month between Minerva finishing and NOvA production
    start. In addition there are some significant modifications to the
    Wide Band Hall that need to be done to accommodate NOvA operations.
    This overall schedule (Minerva/NOvA) seems excessively tight.

-   There are a great many dies/molds that need to be procured for
    production of the varied components (optical connectors, seals,
    covers, etc.) and these parts often have long lead times. The
    production team will have to maintain close coordination with
    procurement in order to guarantee that there are no unexpected
    delays caused by parts delivery delays.

-   The cost estimate is sound given the amount of information
    (engineering drawings, vendor quotes, etc.) that went into the
    B.O.Es. The contingency level is appropriate for this stage of the
    project. The reviewers felt that the contingency on the labor for
    Factory 1 was somewhat low, however.

Recommendations

8.  Fully automate glue application to the end seal extrusion

9.  Reconsider the location of Factory 1 in the Fermilab Wide-Band Hall.
    Consider leasing a facility with enough space to incorporate both
    Factory 1 production activities and the needs for interim storage.

10. Consider improving the fixturing for cutting the 32 cell assemblies
    to length. Even with an edge guide, cutting by hand with a circular
    saw can still lead to an irregular edge. This could lead to sealing
    problems. The current procedure does raise some safety concerns.

11. Prototype and test as soon as possible the baseline method (packed
    desiccant) for insuring that the sealed gas volume surrounding the
    APD (interface region between the APD module and the PVC module
    optical connector) remains dry and prevents any possibility of
    condensation on the APD or fiber surfaces. Some thought should be
    given as to how a dry N~2~ purge could be added if the desiccant
    concept does not work well enough.

12. The team should make use of the evolving 3D model of the detector to
    better evaluate if there will be an interference between vertical
    and horizontal components in some parts of the detector.

13. We recommend that a purchasing expediter be added to the NOvA
    Project Office Staff at the appropriate time.

14. Increase contingency on labor for Factory 1 to 50%

15. Given the potential problems that might occur if the scintillator
    comes in contact with the Devcon PlasticWelder, we recommend that
    the team perform additional tests (hydrostatic) on the barrier seal
    to get an estimate on the expected volume of scintillator that might
    come in contact with the Devcon adhesive.

[]{#__RefHeading___Toc169312693 .anchor}6.0 Electronics, Trigger DAQ
(WBS 1/2.6 & 1/2.7)

Findings

-   Light from the NOvA scintillator will be collected with a segmented
    Avalanche Photodiode (APD) containing 32 pixels that map onto the 32
    cells of a single detector module. A minimum ionizing particle at
    the far end of a detector module (with respect to the readout
    electronics) will yield a signal of 20 photoelectrons. The APD will
    be operated with a gain of \~100. The gain is sensitive to
    temperature and bias voltage variations. The APD will be read out by
    a Front-End Board (FEB) that contains an ASIC that integrates,
    amplifies and multiplexes the signals; a 12-bit ADC to digitize the
    signal; and an FPGA to sparsify and format the data. The FEB also
    will control the Thermo-Electric Cooler (TEC) used to maintain the
    APD operating temperature of --15°C. With these conditions a
    signal-to-noise of 10:1 is expected. Data will be sent from the FEBs
    to 64-channel Data Concentrator Modules (DCM). These collect the
    data in large packets. In order to build events, the 228 Data
    Concentrators use a time-stamp to send data to a single processor
    that is used primarily as a buffer for the data. A farm of
    processors is used, with each processor collecting data in
    one-second intervals. The interconnection between the concentrators
    and the processors will be achieved using commercial gigabit
    ethernet switches. The trigger consists of having the processor farm
    nodes extract data from the buffer memory based on the cycle time of
    the Fermilab Main Injector. Total burdened and escalated costs for
    front-end electronics and the data acquisition system are \$22M and
    \$4.3M, respectively, including 44% and 27% contingencies.

-   The NOvA team is requesting CD3a approval for APD and ASIC
    production and for packaging the ADCs which have already been
    purchased.

Comments

-   The NOvA Front-End and DAQ group presented several well-prepared
    talks during the breakout session. The group has a clear
    understanding of the requirements for the subproject and how to
    achieve them.

-   There has been substantial progress since the Director's CD-1
    review. The prototypes of the APD, ASIC, FEB, and DCM are near to
    the final design and have been validated for much of the needed
    functionality. COTS items (e.g. power supplies, coolers) have been
    identified and costed. There is a well-developed plan for slow
    controls. Prototyping of APD housings is also at an advanced stage.

-   There was no risk analysis for the subproject.

-   There are many discrepancies in WBS. Many of the tasks do not have
    the correct predecessors. Many of the BoEs were not available in the
    BoE book or did not have the correct activity ID corresponding to
    the WBS.

-   The WBS did not include reviews prior to purchase of production
    quantities of custom elements.

-   NOvA has done a buy/build analysis since CD1 and decided to include
    the APD carrier in the contract with Hamamatsu. This is one reason
    for the increase in the expected cost.

-   Data were shown for 15 prototype APDs. Two clearly missed the 5nA
    maximum dark current at 25°C for all channels. 4 had average dark
    current near 4nA, indicating it is likely that the chips did not
    meet the specification.

Recommendations

16. Complete and document a full risk analysis for the APDs. The risk
    analysis should include the possible mitigations for higher dark
    current such as lower operating temperature or changes to FEB
    parameters to enable NOvA to achieve the 10:1 noise specification.

17. A risk analysis is needed for the FEB and DCM. Because the
    production is scheduled to be late in the project, parts may become
    obsolete. It is necessary to consider the relative merits of
    purchasing components early against the possibility of needing to
    redesign the boards.

18. A system integration test including fully functioning APD, TEC and
    FEB is necessary to demonstrate the performance of the APD prior to
    making the purchase of the requested CD-3a items.

19. The WBS needs scrubbed to reflect the updated plan for APD assembly:

-   Remove references to carrier board manufacture

-   Unify nomenclature for "housing", "module", etc.

-   Include module assembly either as a separate step or with testing at
    CalTech and Minnesota.

20. Include reviews prior to purchase of production quantities. The
    approval of the designs and quote packages should be milestones for
    the level 3 subprojects.

21. The WBS should be scrubbed to have proper dependencies. For example,
    the vertical slice tests should depend on completion of testing of
    included devices with a review to follow.

22. The acquisition schedule for the APDs needs to be clarified. The
    vertical slice test was reported to be delayed. The schedule shows
    the APD pilot production for use on the IPND to be occurring 8/07
    through 1/08. Given the uncertainty in APD qualification and the
    negotiations with Hamamatsu, there is some danger that the APDs
    become a critical path item for an IPND run beginning in 10/08.

[]{#__RefHeading___Toc169312694 .anchor}7.0 Far and Near Detector
Assembly (WBS 1/2.8 & 2.9)

Findings

-   The Near Detector is described to be a 215 metric ton mass detector,
    4.2 m x 2.9 m x 16.2 m, fabricated from 496 modules and located in a
    new cavern carved into the side of the MINOS tunnel. The detector
    will include 3 plane block assemblies from the IPND and 28 new
    blocks fabricated at ANL. The blocks will be lowered into the MINOS
    tunnel via an entry shaft and rolled into the new cavern where it
    will be filled with \~30,000 gallons of liquid scintillator. The
    proposed excavated cavern design is not fixed yet and needs the
    installation group to determine access and other technical
    requirements which define the cavern size. The cavern excavation
    will begin at the start of the MINOS downtime in October 2010
    followed by installation of the detector.

-   The Far Detector is an 18kt mass detector fabricated from 1178
    planes (593 V + 585 H) that are developed into 5 Superblocks. Each
    empty superblock is structurally stable and will be filled with
    liquid scintillator when bounded by a second superblock. The total
    volume of liquid scintillator is \~3,900,000 gallons and requires a
    special delivery system piped throughout the Far Detector building.
    The detector assembly group will prepare the hall infrastructure
    including electrical distribution, machine shop, control room,
    office area, shield wall and safety equipment. The assembly group
    will receive \~14,200 completed modules, transport them through the
    gluing stage, and stack them onto a pivot table alternating
    direction to form a 31 plane block unit. The pivot table will tilt
    and install the block unit, weighing \~330,000 lbs, vertically in
    its final location returning to begin assembly of the next block
    unit. Detector plane outfitting will follow including electronics
    racks and chillers, cable trays, chiller water loops, power and data
    distribution boxes. The total assembly process is projected to take
    \~2.8 calendar years beginning after beneficial occupancy of the new
    building in April 2010.

-   Three detector prototypes are planned; the Integration Prototype
    Near Detector (IPND) which is close in size to the Near detector and
    will be fully outfitted with electronics and operated with NuMI
    neutrino beams. The Full-scale Assembly Prototype (FSAP) will be
    \~52' long, assembled of 8 planes and will be used to optimize
    assembly techniques, time and motion studies, adhesive dispenser,
    and handling. The Full-height Engineering Prototype (FHEP) detector
    will be fabricated out of two 31 plane blocks, \~52' long, will be
    filled with liquid, and used to measure deformation over an extended
    period.

Comments

-   Safety considerations are high on the detector assembly engineering
    teams priority list as is evident in the thorough details of the
    Scintillator transfer facility and Block assembly and installation
    planning. This is a good thing.

-   Block assembly and installation has a comprehensive plan detailing
    the labor and equipment needed to meet the project timeline. The
    work plan is for one 10 hour shift per day, 4 days per week, with
    two crews and two days overlap, Monday through Saturday. Breakdowns
    of manpower and task assignments were shown with a hiring plan up to
    the peak of 29 FTE at full production. While the number of
    technicians per task seems set to a minimum, as in 1 FTE for glue
    technician, it was mentioned that the crew bosses are working type
    foremen and will assist when needed. This is an area where a single
    illness or injury could jeopardize the production plan and should be
    evaluated to determine if there is sufficient coverage to sustain
    the full rate assembly. Production fatigue is an area of concern
    both in safety of personnel and in quality of the product; a
    rotation plan for personnel to different tasks may want to be
    considered.

-   Routine maintenance and repair of assembly equipment for the Far
    Detector will be necessary to reduce the MTBF and maintain
    production rate. Development of a planned approach to servicing
    cranes and vacuum lifts (cranes require full load testing on a
    yearly basis), lubrication and maintenance of pivot table, adhesive
    dispenser, as well as the HVAC system required to maintain
    production is warranted and the impact on schedule, if any,
    determined.

-   ES&H activities were clearly shown by the presenters and indicate
    that they are conscious to design safety into their systems as part
    of their engineering planning processes. A NOvA Safety Plan for the
    Far Detector assembly was presented calling for a Safety Committee
    containing members from Fermilab, U of M-ES&H, Detector Safety
    Officer and Detector Manager who would review equipment
    certifications, site inspections and operating procedures --
    excellent plan. However, a "minimum of yearly inspections to insure
    safe working conditions." was one bullet in the slide, and while
    this is important, it is significantly understated. Safety
    inspections and walk-through of a production facility should be done
    on a regular basis, weekly at a minimum, with daily morning tailgate
    meetings with workers to review tasks and procedures. An Integrated
    Safety Management (ISM) plan should be developed to be the core of
    the safety policy, ensuring DOE policies are met.

Recommendations

23. Perform a careful and detailed study of the detector assembly
    program with consideration given to programmatic impacts from normal
    equipment maintenance and worker safety resulting from the project's
    ISM plan. Included in this study should be a review of schedule
    impact due to equipment failure/repair as this will cause an
    alteration to the work planning process. Recommend increasing the
    number of assembly staff at the Far Detector area to allow for on
    going worker training while maintaining a full production
    capability. This can be accomplished without cost increase by
    reduction of the 100% labor contingency.

Technical:

Findings

Near Detector& IPND Assembly:

-   The near detector was not looked at extensively and the IPND was
    looked at less. The proposed cost of the near detector increased by
    about 4.1 million dollars since the last review. This was attributed
    to the fact that now a tunnel will have to be excavated for the
    detector. Across the board, the M&S contingency was listed at 40%
    for all levels and the Labor contingency was listed at 50%.

Far Detector Assembly:

-   NOvA 27 Formulation is relatively new and the creep data is not yet
    obtained. If the committee understands correctly Formulation NOvA 27
    is now the final formulation. The creep simulations are using data
    from PET-B Formulation and calling it a "worst case" scenario.
    Current data for NOvA 27 Formulation is worse than PET-B, but they
    predict that it will be better after 47 days of data are collected
    as seen in the graph below.

![](media/image2.wmf){width="6.010416666666667in"
height="4.114583333333333in"}

Comments

Near Detector& IPND Assembly:

-   It was not found out how the Near Detector contingency numbers were
    assigned but they seem somewhat arbitrary. Perhaps not as much time
    was put into the Near Hall Cost Estimation since the cost is 4 times
    less than the far detector. The scheduling for the near detector
    seemed ok but the excavation time seemed short and the procurement
    of the liquid scintillator filling equipment seemed long as the
    scintillator filling device should be transferred from the IPND. The
    cost and schedule estimates relied on the MINOS experienced staff
    for input.

Far Detector Assembly:

-   If Formulation 27 is not set as the final formulation, then the
    final formulation needs to be set ASAP. The NOvA team should not
    feel comfortable presenting the formulation 27 creep data
    information until at least 30 days of data were calculated. If the
    NOvA 27 Formulation actually has worse creep values than the PET-B
    formulation, then they would almost certainly have to change the
    formulation again. It is the reviewer's feeling that since the
    formulation of NOvA 24 and NOvA 27 are similar in all ways except
    for a change from rutile to anatase TiO~2~, then the creep
    properties will be similar if the particle size of the TiO2 is
    similar for both formulations.

-   Some procedure needs to be implemented that triggers the level of
    review needed for each unit operation. The major items which need to
    be heavily peer reviewed in the far detector are: 1) The Creep
    Analysis 2) The vapor recovery system and the amount of MMA which is
    being produced 3) The block pivoter and 4) The Scintillator
    Distribution System.

-   Some extensive hazard analysis has been done but more still needs to
    be done.

-   Strain gauges need to be specified to see if the detector is
    creeping faster than expected.

-   Minimal risk assessment has been done. Some hazard analysis has been
    done.

Scheduling

-   Scheduling for major components seemed reasonable except for some of
    them did not allow time for design review and none of them allowed
    for change from comments from the review process.

-   It is hard to say how accurate the scheduling is without the risk
    analysis information. The budget did have 100% contingency for labor
    which would allow for 2 shifts to allow for some catch-up. The block
    pivoting device is scheduled under 2.9.1 and 2.9.4, which is
    confusing.

Costing

-   The cost of the far detector went down 4 million dollars since the
    last review and it is most likely from the mass being reduced from
    25 ktons to 18 ktons.

-   One specific error found was in 2.9.1.1, the module lifting fixtures
    had 0% M&S contingency. The labor contingency was also listed as
    100% in the WBS and 85% in the BoE.

Recommendations

Near Detector& IPND Assembly

24. Reexamine the scintillator filling equipment and excavation time
    estimates.

25. Include more information in the BoE quotes including specific
    references.

26. "Scrub" the M&S and Labor contingency estimates.

Far Detector Assembly

27. Set the PVC formulation as soon as possible if it is not already set
    at NOvA formulation 27.

28. Risk assessment needs to be done on all major unit operations.

29. Implement a system that triggers the level of review needed for each
    unit operation and start on major items listed in comments above.

30. The content in the BoE's needs to be improved to include the sources
    that the financial estimates come from and match them to WBS.

Technical (Feasibility and Design status):

Near Detector Structure

Findings

-   A good technical description of a 215 ton near detector sited in a
    newly excavated side cavern is given in the TDR.

-   The module blocks are of identical construction as the far detector
    module blocks except for their overall size (much smaller) and are
    supported from the bottom.

-   The muon catcher modules and steel plates are hung from a steel beam
    support structure. The design of these is currently at about the 15%
    level (mid-conceptual design phase), although the similarity to
    previous MINOS designs indicates no problems.

-   The PVC extrusions will be shipped to FNAL for matching, then
    shipped to UM for assembly, then shipped to ANL for block assembly
    and finally back to FNAL for installation.

-   It is hoped (planned) that 3 of the 4 blocks will be re-used from
    the prototype test in the MINOS service building.

-   Stresses, deflections and creep strains should be minimal due the
    much lower scintillator column height (3 psi head).

-   Common material moving methods will be used to install detector
    blocks (fork truck, wheeled block support frames).

-   Secondary containment is achieved with commercially available
    containment materials and dams.

Comments

-   It should be noted that, due to the similarity to Far Detector
    construction, design assumptions and decisions should be
    re-evaluated during final design of the Near Detector to ensure
    applicability.

Recommendations

None.

Far Detector Structure

Findings

-   The TDR (Chapter 17) adequately describes a current, cohesive
    conceptual design in the advanced state of analysis. On going
    prototypes and mock-ups are being used to verify the validity of
    this analysis. Although a design choice has been made for the
    support structure of the Far Detector, it is clear further
    evaluation and testing are required to gain confidence in that
    choice. Additionally, other support options will be pursued that may
    hold advantages (more economical, more flexible, higher confidence
    in stability).

-   Creep tests using the current pick of PVC formulation (N27) are
    on-going. Further testing is required to confirm the use of the
    "worst-case" (PET-B) creep modulus used in FEA analysis is truly
    conservative.

-   Adhesive peeling force (Devcon) is identified as the limiting factor
    for elastic stresses (as opposed to non-linear creep induced
    buckling) in the structure.

-   Stresses in PVC need to be kept below 600 psi to stay in linear
    viscoelastic range such that using creep modulus test data is valid
    in models.

-   Adhesive is modeled as spring elements and, in the model, does not
    contribute stiffness to the structure.

-   Buckling of the horizontal module webs by block-to-block and
    super-block-to-super-block interactions during creep induced
    vertical block buckling is currently predicted to be the primary
    failure mode at a time-under-load prediction of 20 years (the
    pronounced life-time requirement for the structure).

-   The TDR describes the state of the structure design (as it exists
    now) adequately.

-   The design status is at about 25% for building the described
    detector (if one combines conceptual design as part of the design).

-   Independent peer review of engineering notes and independent
    committee reviews are planned.

Comments

-   We are encouraged that the Project recognizes the need for careful
    review of the structure by outside experts. Selection of expert
    reviewers is important to this process. Also, review committees
    should be charged with assuring consistency of assumptions across
    the various individual system analyses that affect each other.

-   FEA seems to predict accurately the swelling in individual blocks
    and this will be confirmed with planned mock-up tests. FEA of
    interactions between blocks and super-blocks during buckling is
    complicated and not as well understood. One test (2 block test),
    close to full height is planned (with water) that may help gain
    confidence with interactions between blocks. However, the buckling
    will only occur after significant time (5-15 years) and, possibly,
    an upset event, so it is not clear that planned testing will confirm
    those analyses.

Recommendations

31. Continue with current course of FEA analysis and prototype/creep
    testing to explore various loading conditions, sensitivities, and
    verification of analysis. This includes the close to full height 2
    block test with water. The use of water is deemed satisfactory for
    these structural tests. Structural creep tests require long periods
    of data collection and the delay of starting such a test from
    contending with the ES&H and handling issues of large quantities of
    liquid scintillator should be avoided. Chemical interactions between
    PVC and the scintillator oil ingredients can be done in smaller
    stand-alone tests. If interactions are noted, then long term tests
    can be planned and performed at that time.

32. Investigate ways to accelerate block-to-block interaction tests that
    depend on creep and/or creep induced buckling (elevated
    temperature).

33. Plan a rigorous technical review (or series of reviews) of the
    base-line structure design utilizing external experts as soon as
    reasonably possible. This should include experts in the fields of
    plastic creep behavior, non-linear FEA, and buckling.

Far Detector Assembly Equipment (incl. scintillator filling)

Findings

-   Key assembly equipment is in the mid-stages of conceptual design
    (except the module vacuum lifting device which is in the final
    stages of detailed design).

-   The TDR adequately describes the function and conceptual design of
    key equipment (vacuum lifter, glue dispenser, block pivoter,
    scintillator filling equipment).

-   The design status overall of assembly equipment is gauged to be
    about 25%. There exists enough of a conceptual design in most cases
    to enable cost estimates. However, changes to design concepts are
    still possible and/or likely.

-   Design status is not yet advanced enough to perform ergonomic
    studies.

Comments

-   Reliance upon design reviews and peer reviewed engineering notes is
    required.

-   The block pivoter is to be assembled and tested in a "parking lot".
    The weight of a fully loaded pivoter is predicted to be nearly ½
    million pounds. Some thought will obviously be needed to limit the
    testing to what a parking lot can endure.

Recommendations

34. Continue to advance design and prototyping of key assembly equipment
    such that ergonomic and ES&H concerns can be addressed early in the
    design cycle.

Near Detector Assembly Equipment (incl. scintillator filling)

Findings

-   Common material moving methods will be used to install detector
    blocks (fork truck, wheeled block support frames).

-   Secondary containment of scintillator will be provided everywhere
    via PVC "shroud" piping or commercially available dam systems.

Comments

-   None.

Recommendations

None.

Cost, Schedule, Resources, & Risk Management

Near Detector

Findings

-   No BoEs or assigned contingency exists for any R&D activity. If
    contingency is required R&D activities will be de-scoped.

Comments

-   Construction activities (aside from site preparation) begin in
    spring of 2011, it appears that most efforts are not directed at
    these activities (justifiably). As such the only BoE for Mechanical
    Construction and Installation (1 M\$) is the purchase of the Muon
    catcher steel (0.4 K\$).

-   Most of the work (Mechanical Construction and Installation) is using
    conventional, well-understood methods so contingencies (generally
    around 40-50%) are reasonable even though drawings and
    specifications are not completed.

Recommendations

None.

Far Detector

Findings

-   No BoEs or assigned contingency exists for any R&D activity. If
    contingency is required R&D activities will be de-scoped.

-   For construction activities BoE was provided generally at WBS
    level 4. This resulted in 21 BoE's.

-   At least 1 level 4 summary task was supported with a BoE that did
    not cover all aspects of that summary tasks. BoE's for a subset of
    subordinate tasks were also provided. It was not at all clear which
    BoE covered which activities.

-   Schedules provided for Far Detector assembly were based upon
    detailed procedures outlined in the TDR. BoE for assembly work fail
    to reference the TDR and other supporting documentation.

-   BoE's are generally inconsistent in detail and content. Some are
    quite detailed and reference vendor quotes and sections of the TDR
    (and NOvA notes) (such as 2073; WBS 2.9.4.2). Others reference
    "attached" documents that were not provided (2060; WBS 2.9.2.1).
    While others provide little detailed source information at all
    (2057; WBS 2.9.1.2).

-   Contingencies are appropriately high for many tasks since the tasks
    are at the conceptual design phase. 70 to 100 %. Labor cont is 100%
    across the board.

-   Contingency guidance does not allow for more than 100% contingency.

-   Far Detector Assembly has not performed "what-if" scenarios to
    assess schedule impact if key assembly equipment fails.

-   Remote site may incur extra delay penalties when key assembly
    equipment fails.

-   One slide was presented (Block Pivoter) that addressed risk analysis
    by presenting possible failure scenarios with accompanying
    solutions. Otherwise, risk analysis was not presented.

-   Placement of modules into the block assembly has not yet been tested
    on a scale or intensity that supports schedule estimates.

-   Although references to vendor quotes and catalog prices were listed
    in the BoE's, no copies or scanned images of those sources were
    provided.

Comments

-   It is not at all clear that BoE's have been produced at a resolution
    that instills confidence in a reviewer. Our impression is that there
    is much more work that has gone into these tasks and their
    accompanying estimates than is reflected in the BoE's. Being
    engineers that have experienced the "trenches", we understand this
    predicament. However, that may not be the understanding of a DOE
    reviewer.

-   The large number of 100% contingencies listed (especially for labor)
    indicates that the design status is at a relatively low level. While
    this may be accurate, it is of concern that 100% contingency
    actually means equal to or greater than 100% since no contingencies
    above 100% are allowed.

-   The tie between BoE's and OpenPlan numbers is not solid. We found
    instances of discrepancies and confusion (if one adds up the BoE
    totals, minus contingency, it exceeds the cost of the roll-up by
    approx. 1 M\$. And the BoE's are not all inclusive). Also, one BoE
    had listed M&S contingency at 100% where Open Plan listed it as 0%.

Recommendations

35. Provide tie-ins between BoE and technical documents.

36. Increase resolution of BoE's (by writing more BoE's at a lower level
    or showing breakdown of estimates within those BoE's written to
    cover multiple WBS activities; in the latter case, clearly indicate
    on the multiple activity BoE's which activities or portions of
    activities are included).

37. Continue with planned assembly tests and studies (time-motion) to
    provide basis of estimate for schedules.

38. Perform risk analysis and "what-if" analysis to assess schedule and
    cost impact from downtime of key assembly equipment (glue dispenser,
    vacuum lifters, cranes, block pivoter, scintillation filling
    equipment, etc.)

39. Scrub the BoE's and RLS to attain consistency.

40. Write BoE's for R&D activities.

[]{#__RefHeading___Toc169312695 .anchor}8.0 Accelerator Upgrades (WBS
1/2.0.1 & 1/2.0.2)

Findings

-   The scope of the accelerator upgrade portion of the NOVA project is
    to increase the beam power of the 120 GeV beam from the Main
    Injector onto the NuMI target to 700 kW from the present level of
    192 kW.

-   The upgrade will build on the "Proton Plan" that aims at increasing
    the 120 GeV beam power to 320 kW. The required increase in the
    proton throughput of the Booster is planned to be achieved entirely
    as part of the "Proton Plan". The three times increase of the proton
    throughput of the Main Injector (MI) will be achieved by using the
    Recycler Ring (RR) for slip-stacking 12 proton batches from the
    Booster and reducing the MI cycle time to 1.333 s.

-   The conversion of the RR from anti-proton to proton accumulation
    will require 5 new or refurbished kicker systems and two new 53 MHz
    rf systems. The reduced MI cycle time will need two additional 53
    MHz rf systems and an upgrade to one of the quadrupole power
    supplies.

-   Slip-stacking of 11 Booster batches in the MI has been tested with
    the quite good efficiency of about 92%. The beam losses are made up
    of 2.8% from beam in the injection gap, 2.7% of debunched beam, and
    1.7% slow beam loss during the injection front porch. With the
    installation of the gap cleaner kicker and the collimators only the
    latter beam loss is uncontrolled. This is in line with the NOvA
    upgrade design number.

-   The project responded to recommendations from the SNuMI review in
    November 2006. The responses were provided to this committee in a
    document entitled "Table of Recommendations for the Director's
    Preliminary Review of the Super NuMI Plan November 14 -- 16, 2006."
    The project provided responses to six accelerator related
    recommendations that were considered to be related to NOvA. The
    recommendations, the responses and our comments on the responses are
    below.

> **[Recommendation #2]{.underline}**: Concerning the kicker modules,
> their impedance and the danger of electron cloud, the committee
> recommends reconsidering the inside coating of the ceramics in terms
> of resistivity and SEY (Ti, TiN ...).
>
> **Response:** Calculations determined that coating is not required.
> See NOVA-doc-2116. 5/30/07
>
> **Comment:** We note that the referenced document does not
> satisfactorily address the recommendation. However, we are not overly
> concerned about instabilities due to the experimental evidence of
> stable operation in the Main Injector with roughly the same
> intensities and kicker chambers.
>
> **[Recommendation #3]{.underline}**: There seems to be a trade-off
> between the number of bunches "notched" out in the booster and the
> stringent requirements on rise- and fall-time of the injection and gap
> clearing kickers -- the specified 38 ns are based on 2 missing
> bunches. The committee recommends evaluating this trade-off and to
> prepare for a different number of "notched" bunches as a fallback
> solution.
>
> **Response:** Decided to set number of bunches at 81 (of 84) and
> change kicker specifications accordingly.1/15/07
>
> **Comment:** The recommendation has been followed. The results look
> promising.
>
> **[Recommendation #5]{.underline}**: Since slip-stacking to full
> intensity cannot be tested early in the RR, the committee recommends
> continuation of tests in the MI.
>
> **Response:** Agree, will continue aggressively. Progress is shown in
> the MI Upgrades Overview breakout talk. 5/26/07
>
> **Comment:** The committee is satisfied that Fermilab has made
> considerable progress in commissioning multi-bunch slip stacking in
> the MI. The improvement in efficiency is impressive (see comments
> elsewhere in our report). We encourage this work to be continued.
>
> **[Recommendation #6]{.underline}**: Due to the envisaged completely
> new type of operation of the RR without the possibility of relevant
> tests before the end of the Tevatron run, the committee recommends to
> consider at least fully simulating this new operation, including
> longitudinal and transverse beam dynamics.
>
> **Response:** Will address as many aspects as possible with
> simulations. 12/12/06
>
> **Comment:** Progress had been made, and the results are encouraging
> and support experimental data. However, we encourage the continuation
> of this work as well.
>
> **[Recommendation #7]{.underline}**: Concerning the change of BPM
> cables, the committee recommends: Assign a coordinator now who will
> manage the 2009 shutdown activities. Develop the installation plan,
> and examine what activities could be done in earlier shutdowns to ease
> conflicts due to multiple personnel working in the same areas and
> tunnel blockages. (Cables pulls and LCW pipe relocation are two
> obvious candidates for doing early.)
>
> **Response:** A shutdown coordinator will be assigned in a timely way
> for the Accelerator Upgrades shutdown. Planning tasks for this
> shutdown are in the resource loaded schedule. A shutdown plan and
> detailed schedule will be made.
>
> **Comment:** The committee believes this recommendation has not yet
> been followed, and continues to encourage its prompt implementation.
> We note that the recommendation addresses "earlier" (i.e., imminent)
> shutdowns which is the reason that we believe this should be done now.
>
> **[Recommendation #9]{.underline}**: It is unlikely that the losses of
> un-captured beam in the MI will be significantly reduced when 12
> Booster batches are slipped stacked in the Recycler compared to now
> when 11batches are slip stacked in the MI. The collimation system for
> MI must be demonstrated to be effective for Phase I to be a viable
> design for producing 700 kW.
>
> **Response:** The collimator design has been finalized and all the
> collimator parts have been ordered and are expected to arrive by June
> 8th. We expect to have the primary collimator and the four secondary
> ones ready for installation in MI during the shutdown of summer 07.
>
> **Comment:** The response does not address the recommendation, but it
> is a requisite step. In general we were impressed by the loss analysis
> that is presented in the TDR, and therefore the theoretical
> demonstration of the adequacy of the collimator has been shown. This
> recommendation should remain open awaiting the experimental
> demonstration requested in the recommendation.

-   Resources for ES&H coordination and radiological safety for the MI
    and Recycler upgrade have been identified.

-   Plans are to start from the existing Shielding Assessments for the
    RR and MI and update them for NOvA operations.

Comments

-   The technical risk of the proposed hardware items is quite small and
    the presented level of contingency is quite large for items that
    contain little innovation or required R&D. Part of the high
    contingency is a reflection that parts of the design have not been
    frozen. The project should finalize the conceptual design as soon as
    possible and then review the required contingency of the engineering
    designs.

-   The main risk is the control and handling of the beam losses in the
    MI and RR with this greatly increased proton throughput. The recent
    success with demonstrating slip-stacking 4.6e13 protons in the MI
    with 92% efficiency ameliorates this risk substantially. The part of
    the beam loss that would be uncontrolled when the gap cleaner and
    collimator are installed is in line with the design goal for the
    NOvA upgrade. The beam loss from debunched beam that would be
    collected by the collimator is about 2.5 times larger than the
    design. It is likely that this will be reduced through improved
    performance of the Booster or could be covered by adequate margin of
    the collimator system.

-   The NOvA beam power upgrade heavily depends on the success of the
    preceding "Proton Plan" upgrade. Since this constitutes a
    significant risk the project is planning to follow the progress of
    the proton plan and compare performance with realistic simulations
    and further targeted tests at the MI.

-   The scope of the project does not include either commissioning of
    the upgraded facility with beam or system integration testing.
    However, the CD-4 Closeout Definition states that "upgrades and
    modifications are installed and ready for initial operation". This
    requires complete system integration testing and this needs to be
    included in the scope of the project.

-   We performed WBS drill-down exercises in several areas:

```{=html}
<!-- -->
```
-   Drill down #1: RR injection and gap clearing kicker (WBS x.0.1.2.1)

> A prototype kicker is being built under WBS 1.0.1.2.1. The final
> kicker is being built under 2.0.1.2.1. with the power supplies being
> built under 2.0.1.2.2. The cost of the kicker systems is about \$8M
> including power supply. Following the WBS was straightforward and in
> the cases looked at, the BoE tables were consistent.
>
> The long lead time items, the kicker ceramic chambers, are being
> purchased now using Fermilab operating money. They will be tested and
> then put into a spares status. The project will buy them out of spares
> using project dollars when they are needed. To further mitigate the
> risk, the designs have been modified so that all kicker chambers are
> the same cross section.

-   Drill down #2: RR30 Remove Electron Cooling and Rebuild the section
    the FODO lattice

> Again the drill down was straight forward and passed all credibility
> tests that the committee could come up with. This drill down led to a
> look at a number of the other transfer line activities in the recycler
> as these were grouped in the WBS.
>
> Contingencies are relatively high for many of these items which are
> familiar to the Fermilab team doing the work. We discovered that this
> is due to fluidity of the lattice design. Thus the contingency expects
> an increased number of devices and their installation. As soon as the
> design is frozen this analysis should be redone.

-   Drill down #3: 53 MHz system for the RR (WBS 2.0.1.1.2)

> The drill-down was easy and didn't reveal any missing items. Vendor
> quotes were obtained for the major items but no copies were attached
> to the BoE. The cavity fabrication is planned to be done in-house at a
> labor cost of \~\$1M. Outside fabrication could be considered for this
> item. No system integration testing is included. This should be
> included in the required labor resources.

-   Drill down #4: RR BPM cable and board procurement and installation
    (WBS 2.0.1.3.1)

> The major item is a large quantity of heliax cable. This is a catalog
> item with low associated contingency. Again no system integration
> testing is included. This should be included in the required labor
> resources.
>
> In all of these WBS areas the work is Fermilab in house work without
> major procurements that Fermilab has done many times and recently. The
> engineering estimates are therefore based on recent experience.

-   The project is planning to build prototypes of a number of kickers
    including the gap cleaning kicker. The committee concurs that gap
    cleaning is important and NuMI and NOvA would benefit from
    installation of a gap cleaning system in the MI. Whether this could
    be the prototype, or the final magnet built early and then moved,
    should be looked at by the project. Even one year of operation in
    the Main Injector would justify this approach, and give credibility
    to the loss budget established for NOvA.

-   The committee is concerned about the availability of technician
    manpower during the shutdowns, particularly the 8 month shutdown
    scheduled for October 2010. Our concerns relate to the possibility
    of people leaving Fermilab either by retiring, or because of fears
    related to Fermilab's future as the Tevatron shuts down. This is
    something the project will have to monitor and remediate as the time
    approaches. We note this is related to several of the high risk
    factors in the accelerator area.

-   The installation of the MI collimators during the 2007 shutdown will
    require relocating LCW and bus work. This work will be done in such
    a way as to also accommodate the future RR extraction line. Good
    planning! This approach is what is suggested in recommendation 5
    (above), although not thoroughly implemented. The project should
    continue to look for similar opportunities to advance NOvA work into
    earlier shutdowns, and the assignment of shutdown coordinator would
    expedite this.

-   At the time of project base-lining the physics design of the
    accelerator modifications should be frozen and changed only under
    configuration control if needed. This would freeze the component
    count and reduce the contingency in those areas. If possible this
    should be done before the CD-2 review.

-   The committee is satisfied with the projects justifications for the
    CD-3a procurements under this WBS element.

-   The shielding for MI and RR as described in the TDR seem to be
    sufficient for NOvA operations and activation of surface water and
    ground water is expected to remain within the applicable limits.
    Penetrations, labyrinths, drop hatches, will need to be
    re-evaluated.

-   Residual activation of beam line components is an important
    consideration. Beam losses in Booster and MI have been managed with
    good results with determining source terms and addition of
    collimators.

Recommendations

41. Include complete system integration testing in the scope of the
    project.

42. Complete the conceptual design as soon as possible and then review
    the contingencies used in the engineering designs.

43. Consider installing the prototype gap cleaner or the final magnet
    built early in the MI for early testing and use by NuMI. The gap
    cleaner can later be moved to RR.

44. Consider minimizing beam losses in the Recycler as part of the
    design efforts.

[]{#__RefHeading___Toc169312696 .anchor}9.0 NuMI Beamline Upgrades (WBS
1/2.0)

[]{#__RefHeading___Toc169312697 .anchor}9.1 Beamline/TargetModifications

Findings

-   The NuMI upgrades total \$5.3M under WBS 2.0.3. This is split
    roughly equally between M&S and labor.

-   The upgrades are spread over four subtasks: the primary proton
    beamline; the target, baffle and hadron monitor; stripline extension
    and shielding reconfiguration; and utilities.

-   The shielding reconfiguration is the largest subcomponent.

Comments

-   The primary proton beam modifications include moving quadrupoles
    from the A-1 line, power supply upgrades, and improved profile
    monitors. These tasks are all straightforward and carry appropriate
    contingencies of \~25%. Upgrades to the NuMI extraction kicker carry
    a slightly higher contingency of 30%.

-   The major components of the target and baffle are being fabricated
    by IHEP under an MOU. Overall contingency of 47% appears reasonable.

-   The shielding modifications are still under study. Various options
    have been discussed, including restacking the blue blocks (not
    favored due to ALARA concerns); fabrication of a dummy module into
    which T-blocks are inserted (the most expensive option, and the one
    that was costed); a simplified option using different T-blocks but
    no dummy module. This last option appears to be simpler and cheaper,
    and is currently under discussion. The ongoing design effort may be
    an indication of continued difficulty in obtaining sufficient
    engineering and design support.

-   The steel required for the shielding reconfiguration has been costed
    as ordinary steel at \~\$1/lb. A substantial cost reduction would be
    realized if continuous cast salvage steel can be procured at about
    \$0.20/lb.

-   The committee notes that in an ideal world, the NuMI project -
    having already designated three locations for the second horn -
    would have also provided the special modules for these locations,
    thereby avoiding the need to do this reconfiguration in an already
    activated area.

-   The utilities modifications are primarily upgrades of water systems
    to handle the higher power beam. The cost estimates for these appear
    reasonable.

Recommendations

45. Pursue the use of continuous cast salvage steel for the shielding
    reconfiguration. Also see if some or all of the steel might be
    available on site at minimal cost.

[]{#__RefHeading___Toc169312698 .anchor}9.2 Shielding

Findings

-   Radiological safety issues for the NuMI beam line have been
    considered; the areas of concern include: prompt radiation levels
    outside of berm shielding, penetrations, labyrinths, bypass tunnel,
    concentration of radionuclides in groundwater and surface water, air
    emissions and dose due to residual radioactivity.

-   The NuMI Shielding Assessment has been recently updated to 500 kW;
    the project recognizes that it needs to be updated further to
    address NOvA operations.

-   Results from radiation measurements, empirical data and the
    experience gained in operation of NuMI form the basis that the
    project has used to evaluate the radiological conditions that can be
    expected during NOvA operation.

-   The earth shielding thickness for NuMI extraction line is sufficient
    for NOvA operation for normal and accident scenarios.

-   Estimated concentration of radionuclides in groundwater, surface
    water, and dose to air emission are extrapolated from NuMI
    operations and are below applicable limits.

-   Fermilab seems to have made good progress in understanding the issue
    of detectable concentration of tritium in the surface water.

-   Residual activation levels and personnel exposures during the
    shutdowns have been estimated. The activation levels in the target
    hall are already significant for NuMI and will increase for NOvA.

-   NuMI is planning several measures to minimize dose to personnel from
    activated components and materials. These are plans for upgrade of
    Work Cell; and there is a Radioactive Component Removal Plan to
    address short-term storage of components.

Comments

-   While the upgrades in the NuMI target hall are listed as
    off-project, their successful completion is essential for NOvA.

Recommendations

46. Prepare a status report on the issue of tritium in the surface
    water, and discuss why this issue does not pose a risk to the
    project when beam power to NuMI is increased.

[]{#__RefHeading___Toc169312699 .anchor}10.0 Cost

Findings

-   The total project cost estimate (with an 18 kiloton Far Detector) is
    \$297.4 million. This includes \$227.8 million in fully-loaded base
    costs and \$69.6 million (or 31%) in contingency.

```{=html}
<!-- -->
```
-   Construction component: \$217.2 million

```{=html}
<!-- -->
```
-   *Base = \$161.2 million; contingency = \$56 million*

```{=html}
<!-- -->
```
-   OPC component (accelerator/detector R&D, coop agreement, etc.):
    \$80.2 million

```{=html}
<!-- -->
```
-   *Base = \$66.6 million; contingency = \$13.6 million*

-   For reference, the TPC estimate presented at the CD-1 review
    (March 2006) was \$273.4 million, including \$200.9 million in
    fully-loaded base costs and \$72.5 million (or 36%) in contingency.

```{=html}
<!-- -->
```
-   Construction component: \$259.8 million

```{=html}
<!-- -->
```
-   *Base = \$188.6 million; contingency = \$71.3 million*

```{=html}
<!-- -->
```
-   OPC component: \$13.6 million

```{=html}
<!-- -->
```
-   *Base = \$12.3 million; contingency = \$1.3 million*

-   The current estimate includes Accelerator and NuMI upgrades (ANU),
    Accelerator and Detector R&D, and funds for the Cooperative
    Agreement with U-Minnesota to construct the Ash River site and
    buildings. The project team has updated and refined various line
    item costs. Some deliverables (e.g., far detector site and building)
    have been shifted from construction to OPC.

-   The project has a detailed WBS that is used to describe and capture
    the required work. Estimated costs are defined at appropriately low
    levels in the WBS.

-   The project has well-defined contingency analysis rules that are
    applied by the PM, Deputy, and Level 2 managers. Distinct rules are
    defined for labor, M&S and schedule contingency.

-   The project has implemented a Basis of Estimate (BoE) process that
    is used across the project. To date, 328 BoE documents have been
    prepared. Each document is assigned a unique identifier number,
    versioned for configuration control, and stored for easy access in
    the online NOvA document database, Doc-DB. References to BoE
    documents are contained in the WBS. These appear mostly at the
    deliverable level and are intended to provide a quick reference to
    cost basis documentation.

-   The cost estimate has been developed and revised using a combined
    top-down/bottoms-up approach by the project office staff and Level 2
    managers.

Comments

-   The cost estimate was very recently updated by the project; the
    version presented for review was less than 48 hours old. The project
    team acknowledges that it has had limited time to scrub the cost
    estimate for accuracy and completeness. The Project Manager noted
    that through a quick review by "a few people for a few hours", the
    project team has already found approximately \$6.5 million in errors
    that incorrectly increase the project cost.

-   The new cost estimate includes a \$10 million increase to reflect
    new pricing information on two cost elements: the HVAC ventilation
    upgrade for the Far Detector hall and APDs from Hamamatsu. A \$10
    million change based on two line items indicates a certain level of
    volatility in the cost estimate.

-   The project has identified 4 major cost drivers that account for 50%
    of the total project cost. These include the Far Detector Hall,
    Liquid Scintillator, PVC Extrusions, and WLS fiber. Furthermore, we
    heard that small changes can have large effects due to volumes
    required (e.g., \$333K change in project cost for every \$1/barrel
    change in the price of crude oil). We commend the project for
    identifying these large cost drivers and for applying appropriate
    contingency to address the risk. The project should continue to
    monitor these areas, as well as other project areas for new cost
    drivers that may appear as project construction progresses.

-   The review committee spent a short time drilling down through the
    WBS, in an attempt to independently determine accuracy and
    completeness. A number of WBS elements were selected at random for
    the drill-down exercise. In nearly every case, problems were
    identified in the cost estimate. Theses included omissions in costs,
    undocumented cost bases, and links to BoEs that were incorrect. The
    fact that a quick drill-down uncovered numerous problems suggests
    that further work is needed by the project to produce an accurate
    and defensible cost estimate suitable for baseline consideration.

-   In establishing relationships with sole source vendors for PVC
    extrusion production, the project should clearly define ownership of
    tooling (e.g., dies) if the tooling is paid for using project funds.
    This may preclude future cost and schedule issues should problems
    arise in the vendor relationship.

-   A reasonable number of BoEs have been developed by the project and
    used to develop the cost estimate. Given limited time and resources,
    a graded approach was applied to first develop BoEs for the more
    costly items (guidance was given to focus first on items with
    estimated cost \> \$1M). However, a quick sampling of BoE documents
    revealed discrepancies and some duplication (e.g., same information
    contained in the BoEs for WBS elements 2.9.4 and 2.9.4.3). The
    project office should review all BoEs for accuracy and completeness,
    and verify that they are correctly matched to line items in the WBS
    and cost estimate. Recognizing the amount of work required by the
    project staff in a short time period, the committee suggests that
    the project consider seeking additional resources to help with the
    BoE review process. This may free up Level 2 managers to concentrate
    on further scrubbing the cost estimate. Finally, over time, BoEs
    should be developed for all deliverables in the WBS.

-   An overall contingency of 31% for the TPC appears reasonable. A
    quick review of contingency levels in various parts of the WBS
    suggests there may be some inconsistency in the manner in which
    contingency values are being set by the various Level 2 managers. In
    addition, there are concerns that the level of contingency
    established for some subprojects (e.g., Liquid Scintillator and PVC
    Extrusions) may be on the low side given volatility in the commodity
    markets. In the process of scrubbing the cost estimate, the project
    office should review contingency levels for adequacy and
    consistency.

-   The project acknowledges, and the committee concurs, that volatility
    in commodity mineral oil prices imposes a cost risk on the project.
    A reasonable model has been developed and implemented to project the
    cost of mineral oil based on historical data, DOE price projections,
    and industry indices. The current cost estimate is based on a unit
    cost of \$2.98/gallon, based on December 2005 vendor quotes for 6.6
    million gallons. New price data is available as of May 2007, with a
    quoted price of \$3.33/gallon for 3.7 million gallons. The project
    showed that indexing works in the existing model, using 2005 data,
    but the committee suggests updating the model with the more recent
    May 2007 data.

-   The current plan includes two factory assembly operations: a
    Fermilab factory that will perform incoming QA on the PVC extrusions
    and glue together (2) 16-cell extrusions into 32-cell extrusion
    modules; and a Minnesota factory that will install fibers into the
    modules, glue on manifolds, and perform QA (fiber throughput and
    leak tests). The current production model requires the
    transportation of extrusions from the vendor in Manitowoc, WI, south
    to Fermilab, then north to Minnesota. The cost estimate includes 78
    trips south at \~\$4800/trip and 76 trips north at a cost of
    \~\$7100/trip (including contingency), for a total estimated
    transportation cost of \$914,000. Combining the two operations into
    a single factory in Minnesota could result in significant potential
    transportation cost savings, as the extrusions could be shipped
    directly from the Manitowoc vendor to the assembly factory. A
    careful analysis would need to be completed to accurately determine
    the extent of the cost savings, but a quick look at a map and a
    simple back-of-the-envelope calculation suggests that the savings
    may be as high as \$600,000. Combining factory operations might also
    result in further cost savings through economies of scale.

-   The review committee evaluated NOvA's cost and contingency estimate
    spreadsheet and documented their assessment in the notes field of
    the spreadsheet in Appendix A.

Recommendations

47. Complete a thorough and rigorous review of the newly revised cost
    estimate to verify the accuracy and completeness of the June 2007
    estimate prior to baseline consideration.

48. Review the cost basis for each element in the WBS to ensure that 1)
    BoE documents are correctly matched with WBS line items; and 2) each
    BoE contains the appropriate and necessary information to support
    the cost estimate.

49. Verify that peer and independent design reviews are incorporated
    into the WBS at the appropriate level and properly costed.

50. Review the level of contingency assigned to WBS line items by Level
    2 managers to ensure that labor, M&S and schedule contingency rules
    are being applied uniformly across the project.

51. Update the price projection model for mineral oil, using May 2007
    vendor cost data of \$3.33/gallon.

52. Perform and document a cost-benefit analysis to determine the
    potential cost savings of combining PVC module assembly production
    into a single factory site.

[]{#__RefHeading___Toc169312700 .anchor}11.0 Schedule

Findings

-   The schedule presented by NOvA reflects the scope of building and
    installing a 18kton detector at a Total Project Cost (TPC) of
    \$297.4M. The schedule presented for baseline does not meet the DOE
    guidance to not exceed a TPC of \$260M

-   The NOvA presented a Resource Loaded Schedule (RLS) in the
    scheduling software tool Open Plan from Deltek.

```{=html}
<!-- -->
```
-   The schedule presented has 6,126 lines, with 4,394 lowest level
    activities and 474 milestones.

-   The schedule has a total of 89 milestones without a Predecessor or a
    Successor. (20 without a Predecessor and 69 without a Successor)

-   The schedule has a total of 862 activities without a Predecessor
    and/or a Successor. (41 without a Predecessor, 441 without a
    Successor and 380 without a Predecessor or Successor)

-   The schedule has a total of 347 activities and milestones that have
    constraint dates (target dates) assigned. (209 activities and 138
    milestones)

-   The schedule has a total of 1,138 activities that have a duration ≥
    60 work days.

```{=html}
<!-- -->
```
-   CD-4 Milestone completion date is 4^th^ Quarter 2013 with a
    projected schedule completion date of June 2013, which has a float
    of approximately 7 months.

-   Per NOvA's Preliminary Project Execution Plan (PPEP) and Preliminary
    Project Management Plan (PPMP), a tiered milestone system is to be
    used, which includes Critical Decision, Level 1, 2 and 3 milestones.
    The NOvA schedule has 474 milestones with 319 milestones without the
    tier identified.

-   The NOvA schedule does include some design review activities. NOvA's
    Project Engineer has a plan for engineering design reviews, which
    needs to be uniformly implemented and included in the schedule.

-   Schedule contingency is being applied by building it into specific
    milestones and then monitoring the float. The schedule contingency
    process is identified in the Contingency Analysis Rules for NOvA
    document section 2.5.

-   Some Resource Leveling has been performed on the NOvA schedule.

-   No one-page master schedule for the entire project was available or
    a one-page schedule for the individual subprojects including the
    critical path.

-   328 Bases of Estimates (BoEs) were available at the time of the
    review. Guidance for initial generation of BoE's was to concentrate
    on items \$1M or greater. The BoE's for the Accelerator & NuMI
    Upgrades subproject have been completed and the other subprojects
    are at different stages of BoE development. A standard form for
    documenting the BoE was used and the completed forms ,with
    associated reference documents, are stored and controlled in NOvA's
    docdb repository. Binders with BoEs were available during the
    review. Some NOvA team members saw the BoE binders for the first
    time in the review breakout sessions. NOvA plans to continue to
    create BoE's for the other activities that do not presently have
    one. There are some existing BoE's that need the supplemental
    documentation and the numbers (hours/dollars) need to be reconciled
    between the BoE and the schedule.

-   The WBS Dictionary exists in the Draft Technical Design Report (TDR)
    Section 7 for Level 2 and 3 of the WBS. WBS Definitions exist in the
    NOvA Open Plan Schedule that gives more detail than the WBS
    Dictionary. These definitions are at varies levels of the WBS from
    Level 3 to Level 6.

-   Milestone Definitions exists in the NOvA Schedule for the
    Accelerator & NuMI Upgrades subproject but not for the other
    subprojects.

-   A draft NOvA Risk Management Plan document exists and per the NOvA
    Deputy Project Manager it has been partially implemented. The
    Accelerator & NuMI Upgrades and Far Detector Site & Building
    subprojects have completed the more detailed bottom-up risk
    analysis. The other subprojects have performed a top-down risk
    assessment that identified the high-level key risks. The project
    recognizes that the more detailed bottom-up risk analysis needs to
    be done by the Level 2 Managers. The currently identified risks are
    recorded in a Excel Risk Registry and will be migrated into
    WelcomRisk software by Deltek.

Comments

-   The NOvA project recognizes that the schedule needs to be scrubbed
    to validate the content since the schedule was completed less than
    48 hours before the start of the review. The review committee agrees
    with that assessment. Some of the obvious areas that need to be
    scrubbed are as follows:

```{=html}
<!-- -->
```
-   Schedule mechanics which includes appropriately assigning
    predecessors and successors to activities and milestones and
    minimize the number of constraints (target dates) used. With the
    lack of predecessors/successors and having constraint dates the true
    detailed critical path may not be reflected in the schedule.

-   Evaluate the long duration activities to determine if they can be
    broken up into small duration tasks with specific deliverables or
    that there are adequate existing milestones to measure the long
    activities' progress. This is import to improve accuracy of status
    and measuring progress. Also, it is critical when determining on how
    to earn the value for Earned Value Reporting.

-   Evaluate if the appropriate number of milestones exists so progress
    can be monitored and assign the milestone tier level.

-   Evaluate if the engineering review activities exist in the schedule
    and determine if additional ones need to be added and resource
    loaded.

-   Utilize WBS Definitions notes field that exists in the schedule to
    further define the lower level WBS activities. This supplements the
    high-level WBS Dictionary.

-   Complete Milestone Definitions for all the project milestones.

-   Incorporate any activities and associated cost that result from
    mitigation plans that are identified after the more detailed
    bottom-up risk assessment has been completed.

-   After completing scrubbing the schedule for the items listed above,
    assess the resource need vs. the availability of resources and
    perform resource leveling where needed. This helps in increasing the
    likelihood that the activities can be completed in the timeframe
    scheduled and minimize variances in Earned Value Reporting.

```{=html}
<!-- -->
```
-   In addition to scrubbing the existing schedule for errors, revising
    the schedule to bring the TPC to meet the OMB guidance, not to
    exceed \$260M, is a significant effort on its own.

-   The schedule that shows the critical path for entire project was not
    available to the committee via the webpage, but one was generated
    when requested. A detailed critical path for the entire project and
    by subproject should be made available to the review committee ahead
    of the review. Additionally a one page master schedule with critical
    path should be generated and available to the review committee.

-   The NOvA project has created many more BoEs since the Director's
    CD-1 Review. The standard BoE form that is now being used has
    greatly improved consistency of the type of information to be
    included in the BoEs. Additional BoEs still need to be generated and
    existing BoEs need to be validated to insure that the BoEs and the
    schedule match.

-   The content of the WBS Dictionary appears to be sufficient at Level
    2 and 3 of the WBS. It is strongly suggested that the WBS Definition
    notes field are utilized in the schedule to describe the activities
    at lower levels of the WBS be more consistent. This type of detail
    helps to insure that there is a common understanding of the work to
    be performed by the person that generated the activities and the
    person that has to do the work. It also helps when the person that
    generated the activities is no longer around to explain what was
    meant by the short activity description.

-   The use of milestone definitions that exist in the schedule for the
    Accelerator & NuMI Upgrades subproject should be applied to the
    entire project. By defining and documenting what action(s) need to
    be completed to status the milestone as complete will be more
    clearly understood by more people than the person that generated the
    milestone. This is especially important when milestones are being
    used to monitor project progress and especially if the completion of
    the milestone triggers to earn value as part of an Earned Value
    Management System (EVMS).

-   The current implementation of Risk Management by NOvA appears to be
    following NOvA's Risk Management Plan, but at the current baseline
    stage a complete risk assessment with mitigation plans needs to be
    completed for the entire project. This includes a bottom-up and the
    top-down assessment for all parts of the project.

Recommendations

The following recommendations are to be completed prior to the DOE
CD-2/3a Review.

53. The schedule is to be scrubbed to reduce cost, including modifying
    the detector size, in order to meet the OMB guidance not to exceed a
    TPC of \$260M.

54. To insure schedule quality, the schedule is to be scrubbed for the
    items listed in the first Comment of the schedule section (11) of
    this report.

55. Complete the detailed bottom-up risk assessment for the entire
    project, which includes mitigation plans. Incorporate the mitigation
    activities in the cost and schedule where appropriate.

56. Assess at which level(s) in the WBS the BoEs should be generated.
    Complete all the BoE's for the project. Compare the content of the
    BoEs with the data in the schedule and insure they match (Similar
    Recommendation issued from the Director's CD-1 Review of NOvA).

57. Generate and maintain a one-page master schedule with the critical
    path.

[]{#__RefHeading___Toc169312701 .anchor}12.0 Project Management (WBS 1.9
& 2.10)

Findings

-   An overview of the project and cost drivers was given by project
    management. With the inclusion of the ANU subproject there are now
    10 fully staffed subprojects, including the one for project
    management, which has a cost of \$5.7M.

-   The project presented the review committee with the documents
    required for DOE CD-2 review

```{=html}
<!-- -->
```
-   TDR, EA, AS, QA, RM, etc.

```{=html}
<!-- -->
```
-   The PPEP and PPMP needed for CD-1 were made available

-   No single page master schedule with critical path was shown for
    either the overall project or the subprojects.

-   Formal change control is planned but not currently implemented.

-   Earned value reporting not yet implemented in monthly reports, which
    have been done for some time in narrative form.

-   No draft MOU's or SOW's were shown to the committee

-   The project office staffing is projected to be 7.5 FTE's in FY'08,
    and 8.0 FTE's in FY'09-FY13. There is no contingency on PO labor,
    and no costs included for external expertise for consulting or
    reviews.

-   No response to previous directors CD-1 review recommendations was
    shown.

Comments

-   The project should be commended for their continued rapid progress
    towards baselining the NOvA project. The project team is a strong
    and experienced one, and they have all the tools in place to
    successfully manage the project. There are many requirements for a
    baseline review, and it is clear the team is aware of their
    responsibilities and has worked hard to meet them. The Technical
    Design Report and other project management documents are largely
    complete, but will need additional efforts to make them ready for a
    baseline review. The project team presented their materials in a
    relatively unified format, and the website organization was a
    significant improvement over previous reviews. Unfortunately, the
    review team was somewhat hindered in preparations for the review by
    many of the review materials only becoming available the morning of
    the review. This will probably not be acceptable for a DOE baseline
    review. The production of monthly reports on a routine basis is
    excellent, and needs only the addition of earned value reporting to
    be in the final format needed for the duration of the project. The
    project team did an excellent job of answering all the reviewers'
    questions.

-   Management of the Ash river construction has dual authority lines in
    the organization chart shown. Project planning depends on an MOU
    between Fermilab and the University of Minnesota which has not yet
    been drafted. Responsibilities of the Project Manager and the
    Federal Program Manager would benefit from additional clarification.
    Fermilab, project management, University of Minnesota, and DOE roles
    and responsibilities, especially with regards to safety and
    contingency utilization, would benefit from such an MOU.

-   The PPEP scope range showing a 20kt detector and a 6 year run should
    be updated as needed and developed into a PEP prior to any baseline
    review. The DOE Dep. Project Director is shown in the text but is
    not in the organization chart, and that should be corrected.

-   The PPMP should be updated to reflect the current proposed baseline,
    references to the PMSD should be deleted, and it should be developed
    into a PMP prior to a baseline review.

-   It is important to put at least draft MOU's and SOW's in place soon.
    Understanding of the resources being planned on by the project, and
    committed by the collaborating institutions (including Fermilab)
    will avoid misunderstandings and put the project labor planning on
    more solid ground.

-   The Environmental Assessment covers both Fermilab and Minnesota and
    includes information for up to 1.5 MW of beam power. Figure 4.2 says
    NuMI tunnel is 650ft below mean sea level, which does not sound
    right.

-   The Risk Management Plan is appropriate for this project, but the
    implementation should rapidly become more complete. The Risk
    Registry formulation is adequate, and the migration to WelcomRisk is
    a good move. The Level 2 managers have not had time to complete risk
    assessments for their subprojects.

-   The Project Engineer has a good plan for engineering design reviews,
    which need to be uniformly communicated to lower level managers and
    shown in the RLS. Some funding in 2.10 will probably be needed to
    pay for these kinds of reviews if they are to be successful and
    widely used.

-   The Project has a large number of procurements and is working
    closely and well with Fermilab's procurement people. The addition of
    an expediter in the project office to work with them and the project
    might reduce schedule slippages.

-   It was observed that there is inconsistency on how final design
    activities are handled between detector and ANU parts of the
    schedule. The activities can be found in the either WBS 1 (R&D) or 2
    (Construction) of the schedule and may be R&D or MIE funds. Also, it
    was not clear if and how contingency was applied to these
    activities. The project should evaluate and address any
    inconsistencies with where the design activities are located in the
    schedule, funding type and the application of contingency and/or be
    quite explicit when describing these final design activities for
    various part of the project.

Recommendations

58. It is important to have a draft MOU begun between Fermilab and the
    University of Minnesota to supplement the CA.

59. Prepare draft MOU's and SOW's for institutions planning on doing
    work for the project so that responsibilities and labor resources
    commitments are clearly understood.

60. Finalize both PEP and PMP prior to a baseline review.

61. There needs to be detailed balance of funding and cost by FY by the
    baseline review in the project management documentation.

62. The project office needs to work with the L2 managers to complete
    the implementation of the Risk Management Plan.

63. Complete the TDR and scrub all CD-2 related documentation to reflect
    the current project status and scope.

64. Consider adding an expediter to the Project office.

65. Include costs in WBS 2.10 for external reviewers/consultants, labor
    contingency, and an expediter.

66. Preparation of materials for the baseline review should begin early
    enough that the reviewers have adequate time to prepare for the
    review.

[]{#__RefHeading___Toc169312702 .anchor}13.0 Charge Questions

Technical

1.  Are the technical specifications clearly stated and documented?

The technical specifications are clearly stated and documented in the
TDR.

2.  Can the design be built? Does the design meet the technical
    specifications? Is it a reasonable design?

The accelerator and NUMI upgrade design meets the technical
specification, is reasonable and can be built.

The detector technology chosen is reasonable, very well optimized and
meets the technical specifications, with the exception of the fiducial
mass for the far detector which is driven by cost considerations.

3.  Does the baseline design meet the project's objectives (mission
    need)?

The baseline design meets the project objectives, however we note that
cost driven reductions in the fiducial mass have decreased the physics
sensitivity by 20-30% unless the running time is increased beyond six
years.

Cost

4.  Is the Work Breakdown Structure (WBS) appropriate for the project
    scope?

The WBS is well-developed and broad in scope. It encompasses all
significant project activities and appears to have an appropriate level
of detail to accurately develop the project cost estimate and a
realistic timeline for project completion; and to establish a baseline
against which to track future performance. The WBS is still undergoing
active development as the cost estimate is revised and scrubbed.

5.  Do the cost estimates for each WBS (or cost) element have a sound
    documented basis and are they reasonable?

The project is still in the process of documenting the cost basis for
all WBS elements. At the time of the review, Basis of Estimate (BoE)
documents had been developed for 328 of the WBS elements. These BoEs
contain information such as assumptions, labor estimates, commodity
price projections, and vendor quotes. The project has taken a graded
approach in documenting the basis for the cost estimate, by initially
focusing on generating BoEs for items with estimated costs of \$1M or
more. BoEs for some subprojects (Accelerator and NuMI Upgrades) are
substantially complete. BoEs for other subprojects are in various stages
of development. There are no BoEs for remaining R&D activities. The
information contained in many of the existing BoEs seems reasonable, but
further scrubbing by the project is needed to validate the accuracy of
the information in the BoEs, to verify that the information is
sufficient to provide a reasonable basis for the cost, and to verify
that they are accurately mapped to deliverables in the WBS.

6.  Does an obligation profile exist? How does it compare with the
    funding guidance?

The Project Manager presented the current cost profile for the project. 
The cost profile shows projected costs on an annual basis, compared to
the DOE funding profile. Project costs through FY10 fall within the
funding guidance profile. Projects costs in FY11-13 exceed the funding
guidance profile. This reflects the fact that the current TPC estimate
of \$297.4 million exceeds the DOE funding guidance to not exceed a TPC
of \$260 million.

Schedule

7.  Is the schedule well developed and appropriately structured by
    specifying relationships, predecessors, successors, critical path,
    resource loaded, etc?

The schedule is well developed for a TPC of \$297.4M but it can not be
assessed for a schedule that is required to have a TPC of ≤ \$260M. The
schedule needs be scrubbed to correct some structural issues and a
detailed bottom-up Risk Analysis completed before it can be determined
if the schedule is adequately developed.

8.  Are the durations for the activities and overall schedule reasonable
    and achievable with the assumed resources?

One forth of the activities has duration's ≥60 work days, which is not a
desired state. The project needs to evaluate if the activities can be
broken down to small tasks with specific deliverables or if adequate
milestones exist that can demonstrate progress. After completion of the
detailed bottom-up risk assessment is completed and changes incorporated
and the schedule is scrubbed to meet the \$260M TPC guidance, an
assessment can then be performed to say if the schedule is reasonable
and achievable.

9.  Does the schedule contain appropriate levels of milestones,
    sufficient quantity of milestones for tracking progress and do they
    appear to be achievable?

There are 474 milestones in the NOvA schedule. By shear number of
milestones sufficient quantities may exists for tracking progress, but
since 319 milestones have not been assigned as a Level 2 or 3 tier the
analysis could not be completed.

10. Does the schedule include activities for design reviews, including
    assessment of the design's readiness for procuring prototypes,
    preproduction and production materials?

The schedule does include some activities for design reviews, but the
Project Engineer has a plan for engineering design reviews, which need
to be uniformly implemented and included in the schedule.

Management

11. Is there an appropriate management organizational structure in place
    to accomplish the design and construction?

The management structure is complete, and should be adequate to manage
the design and construction. The University of Minnesota management of
the Ash River site was not looked into in detail, but they have
extensive experience in construction projects, and we assume that if the
roles are responsibilities are clearly defined that successful
completion of that part of the project should be accomplished.

12. Is the organization structure well documented, responsibilities
    defined and appropriate for the scope of work?

Yes

13. Are there adequate staffing resources available or planned for this
    effort?

Yes, addition of an expediter to the project office would be beneficial.

14. Is there a funding plan available or proposed to meet the resource
    requirements to realize the project?

No, the funding profile does not match the resource requirements
presented by the project

15. Has a Risk Plan been developed, risks identified, risks analyzed,
    risk responses planned/implemented, risk monitoring/control process
    established and do they seem appropriate?

There is a risk management plan in place, but it is not fully
implemented by the project.

Procurement

16. Have the critical procurements been identified and are they included
    in the schedule with adequate lead time built in?

Critical Procurements have been identified. However the terms procure,
purchase, etc. are used to identify different sets of activities in the
procurement process. In some cases it appears that the activity includes
production or delivery. Standard procurement milestones should be
employed for consistency

17. Have critical make vs. buy decisions been evaluated in conjunction
    with the scope and is that reflected in the baseline cost estimate,
    schedule and technical risk plan?

Since the last director's review the decision to change strategy from
in-house scintillator mixing to contracting for the service was made. Is
it possible to have the extruder undertake the initial assembly tasks of
gluing 2 16-cell extrusions together and final end cuts at the extrusion
factory before skidding/wrapping/shipping?

18. Are the Project designs final and procurement packages prepared to
    the degree appropriate to order materials and initiate construction
    as scheduled?

All of the major commodities have been ordered in small batches, and the
formulations adjusted to a degree that they can be contracted for.
Procurement plans should be developed for each major procurement which
specifies procurement milestones, acquisition strategy (competitive/sole
source), phased funding, options, blankets with delivery orders, sources
etc. The construction (3a) procurements are sufficiently defined, while
designs for equipment for far detector assembly has not been completed.

[]{#__RefHeading___Toc169312703 .anchor}Appendices

Cost Estimates

Charge

Agenda

Report Outline and Reviewer Writing Assignments

Reviewer Assignments for Breakout Sessions

Reviewers' Contact Information

Participant List

Table of Recommendations

Appendix A

[]{#__RefHeading___Toc169312704 .anchor}Cost Estimates

NOvA's Cost Estimate for the Director's CD-2/3a Review of the NOvA
Project

June 4 -- 6, 2007

![](media/image3.wmf)

Review Committee's Cost Estimate for the Director's CD-2/3a Review of
the NOvA Project

June 4 -- 6, 2007

![](media/image4.wmf)

Appendix B

[]{#__RefHeading___Toc169312705 .anchor}Charge

for the Director's CD-2/3a Review of the NOvA Project

June 4 -- 6, 2007

This charge is for the Committee to conduct a Director's CD-2/3a Review
of the proposed NOvA project at Fermilab. The review is to assure that
all the requirements will be met for DOE to approve CD-2/3a. The DOE
CD-2/3a review is currently scheduled for July 17-19, 2007.

The purpose of the NOvA project is to fabricate the NOvA near and far
detectors and to provide a detector hall for the far detector, as well
as upgrade the Fermilab Recycler and Main Injector accelerators and the
NuMI beamline. The ensemble will permit the experimenters to study
neutrino oscillations, in particular, to search for the oscillation of
muon-type neutrinos to electron-type neutrinos. If these oscillations
can be observed then the experimenters may be able to determine the
mass-ordering of the neutrinos and to observe Charge Parity (CP)
violation in the neutrino sector. Determination of the mass-ordering is
a unique contribution made possible by NOvA's very long baseline.

CD-2 is approval of the Performance Baseline. The Performance Baseline
is developed based on a design document (Preliminary Design or a
Technical Design Report), a well-defined and documented scope, a
resource-loaded detailed schedule, a definitive cost estimate, defined
Key Performance Parameters and some additional project management
documents. Approval of CD-2 authorizes submission of a budget request
for the Total Project Cost (TPC) and detailed engineering design.

CD-3a is approval to start limited Construction. NOvA is requesting
CD-3a for infrastructure and site preparation work to support the start
of building construction and limited items for the Detector and ANU
activities that are either long lead time items or parts required to
start construction of critical items. The design and engineering for
these items should be completed to the degree appropriate to initiate
construction as scheduled. A review of the CD-3a items should be
performed to assure that all environmental, safety and security criteria
are met. DOE CD-3a approval provides authorization to complete
procurement and construction of the specified work.

The technical part of the review should focus on the designs for the
detector and building as well as the upgrades to the accelerator and
NuMI. Respond as to whether the designs meet the technical
specifications and whether the designs are sound. The cost and schedule
baselines are based on a detailed WBS -- Work Breakdown Structure, WBS
Dictionary, BoE -- Basis of Estimate documentation, risk and contingency
analyses, RLS -- Resource Loaded Schedule, and time phased funding and
cost profiles. The committee is asked to review each of these items, for
quality, completeness, and accuracy. The committee is also asked to
review and assess the quality of and comment on the additional formal
project management documentation provided in support of CD-2/3a.

DOE's guidance to NOvA is to not exceed a Total Project Cost (TPC) of
\$260M. Based on the scope of work presented during the review, the
committee is to assess whether the project can be built within the
guidance. If it is determined that the work scope as presently defined
cannot be completed within the guidance, then the Far Detector mass will
be the relevant scope parameter.

As part of this assessment the questions listed in Attachment 1 of this
charge should be addressed. Additionally the review committee is to
review and comment on the Project's response and actions taken on the
recommendations from the Director's CD-1 Review of NOvA on February 28 -
March 2, 2006 and from the DOE CD-1 Review conducted April 4-6, 2006.
The review committee is to also review and comment on the Project's
response and actions taken on the relevant ANU related recommendations
from the Director's Preliminary Review of the Super NuMI (SNuMI) Plan
conducted on November 14-16, 2006. Constructive comments on presentation
content, format, and style are also requested.

Finally, the committee should present findings, comments, and
conclusions at a closeout meeting with NOvA's and Fermilab's management
and provide a written report soon after the review.

Charge for the Director's CD-2/3a Review of the MINERvA Project

Attachment 1

Technical

-   Are the technical specifications clearly stated and documented?

-   Can the design be built? Does the design meet the technical
    specifications? Is it a reasonable design?

-   Does the baseline design meet the project's objectives (mission
    need)?

Cost

-   Is the Work Breakdown Structure (WBS) appropriate for the project
    scope?

-   Do the cost estimates for each WBS (or cost) element have a sound
    documented basis and are they reasonable?

-   Does an obligation profile exist? How does it compare with the
    funding guidance?

Schedule

-   Is the schedule well developed and appropriately structured by
    specifying relationships, predecessors, successors, critical path,
    resource loaded, etc?

-   Are the durations for the activities and overall schedule reasonable
    and achievable with the assumed resources?

-   Does the schedule contain appropriate levels of milestones,
    sufficient quantity of milestones for tracking progress and do they
    appear to be achievable?

-   Does the schedule include activities for design reviews, which
    include assessment of the designs readiness for procuring
    prototypes, preproduction and production materials?

Management

-   Is there an appropriate management organizational structure in place
    to accomplish the design and construction?

-   Is the organization structure well documented, responsibilities
    defined and appropriate for the scope of work?

-   Are there adequate staffing resources available or planned for this
    effort?

-   Is there a funding plan available or proposed to meet the resource
    requirements to realize the project?

-   Has a Risk Plan been developed, risks identified, risks analyzed,
    risk responses planned/implemented, risk monitoring/control process
    established and do they seem appropriate?

Procurement

-   Have the critical procurements been identified and are they included
    in the schedule with adequate lead time built in?

-   Have critical make vs. buy decisions been evaluated in conjunction
    with the scope and is that reflected in the baseline cost estimate,
    schedule and technical risk plan?

-   Are the Project designs final and procurement packages prepared to
    the degree appropriate to order materials and initiate construction
    as scheduled?

Appendix C

[]{#__RefHeading___Toc169312706 .anchor}Agenda

for the Director's CD-2/3a Review of the NOvA Project

June 4 -- 6, 2007

Monday, June 4, 2007

  ----------------- ---- --------------------------------- ----------------
  8:00 -- 8:45 AM   45   Executive Session (Comitium,      Ed Temple
                         WH2SE)                            

  9:00 -- 9:10 AM   10   Welcome and Laboratory Overview   Hugh Montgomery
                         (Hornets Nest - WH8X, Overflow in 
                         Racetrack -- WH7X)                

  9:10 -- 9:55 AM   45   Project Overview                  John Cooper

  9:55 -- 10:15 AM  20   Project Cost Drivers              Ron Ray

  10:15 -- 10:45 AM 30   Accelerator and NuMI Upgrades     Nancy Grossman

  10:45 -- 11:00 AM 15   **BREAK** (Outside Hornets Nest - 
                         WH8X)                             

  11:00 -- 11:30 AM 30   Site and Building                 Steve Dixon

  11:30 -- 11:50 AM 20   Scintillator                      Stuart Mufson

  11:50 -- 12:00 PM 10   Fiber                             Carl Bromberg

  12:00 -- 12:20 PM 20   PVC and Extrusions                Rich Talaga

  12:20 -- 1:20 PM  60   **LUNCH** (WH2 Crossover)         

  1:20 -- 1:40 PM   20   Extrusion Modules                 Ken Heller

  1:40 -- 2:05 PM   25   Electronics and DAQ               Leon Mualem

  2:05 -- 2:40 PM   35   Near/Far Detector Assembly        Dave Ayres

  2:40 -- 2:55 PM   15   NOvA Science & Detector           Mark Messier or
                         Performance                       Gary Feldman

  2:55 -- 3:10 PM   15   **BREAK** (Outside Hornets Nest - 
                         WH8X)                             

  3:10 -- 4:25 PM   75   BREAKOUT SESSIONS                 

                         1\) [Site and                     Steve Dixon\*
                         Building]{.underline}             
                         (Confessional -- WH5NE)           

                         2\) [Commodities - Scintillator,  Rich Talaga\*
                         Fiber, PVC]{.underline} (Snake    
                         Pit -- WH2NE)                     

                         3\) [Far and Near Detector        Dave Ayres\*
                         Assembly]{.underline} (The Req.   
                         Room -- WH4NW)                    

                         4\) [Electronics and              Leon Mualem\*
                         DAQ]{.underline} (Hornets Nest -  
                         WH8X)                             

                         5\) [Extrusion Module             Ken Heller\*
                         Production]{.underline} (Black    
                         Hole -- WH2NW)                    

                         6\) [Accelerator and NuMI         Nancy Grossman\*
                         Upgrades]{.underline} (Racetrack  
                         -- WH7X )                         

  4:30 -- 6:30 PM        Executive Session (Comitium,      
                         WH2SE)                            
  ----------------- ---- --------------------------------- ----------------

Tuesday, June 5, 2007

  ------------------ --- ---------------------------------- --------------
  8:00 -- 8:30 AM        Cost and Schedule Executive        Ed Temple
                         Session (Comitium, WH2SE)          

  8:30 -- 8:45 AM        Cost and Schedule Methodology      Bill Freeman
                         (Comitium, WH2SE)                  

  8:45 -- 10:45 AM       BREAKOUT SESSIONS                  

                         1\) [Site and                      Steve Dixon\*
                         Building]{.underline}              
                         (Confessional -- WH5NE)            

                         2\) [Commodities - Scintillator,   Rich Talaga\*
                         Fiber, PVC]{.underline} (Snake Pit 
                         -- WH2NE)                          

                         3\) [Far and Near Detector         Dave Ayres\*
                         Assembly]{.underline} (The Req.    
                         Room -- WH4NW)                     

                         4\) [Electronics and               Leon Mualem\*
                         DAQ]{.underline} (Hornets Nest -   
                         WH8X)                              

                         5\) [Extrusion Module              Ken Heller\*
                         Production]{.underline} (Black     
                         Hole -- WH2NW)                     

                         6\) [Accelerator and NuMI          Elaine
                         Upgrades]{.underline} (Racetrack   McCluskey\*
                         -- WH7X )                          

                         7\) [Cost, Schedule and            John Cooper\*
                         Management]{.underline} (Comitium, 
                         WH2SE)                             

  10:45 -- 11:00AM       **BREAK** (Outside Comitium,       
                         WH2SE)                             

  11:00 -- 12:45 PM      BREAKOUT SESSIONS -- Continued     
                         (Same breakouts and locations as   
                         for the 8:45 -- 10:45 AM sessions) 

  12:45 -- 1:45 PM       LUNCH (WH2 Crossover)              

  1:45 -- 2:45 PM        NOvA Respond to Committee          
                         Questions from 1^st^ Day           
                         (Comitium, WH2SE)                  

  2:45 -- 6:30 PM        Executive Session and Report       
                         Writing (Comitium, WH2SE) Breaks   
                         taken as necessary.                

                                                            

  Wednesday, Jun. 06                                        

  8:00 -- 9:30 PM        Subcommittee Working Sessions and  
                         Report Writing                     

  10:00 -- 2:00 PM       Committee Closeout Dry Run with    
                         working lunch (Comitium, WH2SE)    
                         Breaks taken as necessary.         

  2:00 PM                Closeout ((Hornets Nest - WH8X,    
                         Overflow in Racetrack -- WH7X)     
  ------------------ --- ---------------------------------- --------------

\* Notes Breakout Session Lead

Appendix D

[]{#__RefHeading___Toc169312707 .anchor}Report Outline and Reviewer
Writing Assignments

for the Director's CD-2/3a Review of the NOvA Project

June 4 -- 6, 2007

+------------------------------------------------------+---------------+
| Executive Summary                                    | [Ed           |
|                                                      | Temple        |
|                                                      | ]{.underline} |
+------------------------------------------------------+---------------+
| 1.0 Introduction                                     | [Dean         |
|                                                      | Hoffer        |
|                                                      | ]{.underline} |
+------------------------------------------------------+---------------+
| 2.0 Science                                          | [Heidi        |
|                                                      | Schellman]    |
|                                                      | {.underline}, |
|                                                      |               |
|                                                      | and All       |
+------------------------------------------------------+---------------+
| 3.0 Site and Building (WBS 1/2.1)                    | [Karen        |
|                                                      | Hellman]      |
|                                                      | {.underline}, |
|                                                      |               |
|                                                      | Jeff Sims     |
+------------------------------------------------------+---------------+
| 4.0 Commodities -- Scintillator/Fiber/PVC (WBS       | [Linda        |
| 1/2.2, 1/2.3 & 1/2.4)                                | Stutte,       |
|                                                      | ]{.underline} |
|                                                      |               |
|                                                      | Joe Ingraffia |
+------------------------------------------------------+---------------+
| 5.0 Extrusion Module Production (WBS 1/2.5)          | [Alan         |
|                                                      | Bross,        |
|                                                      | ]{.underline} |
|                                                      |               |
|                                                      | Heidi         |
|                                                      | Schellman     |
+------------------------------------------------------+---------------+
| 6.0 Electronics, Trigger DAQ (WBS 1/2.6 & 1/2.7)     | [Jonathan     |
|                                                      | Lewis]        |
|                                                      | {.underline}, |
|                                                      |               |
|                                                      | Eric James    |
+------------------------------------------------------+---------------+
| 7.0 Far and Near Detector Assembly (WBS 1/2.8 & 2.9) | [Richard      |
|                                                      | Boyce]        |
|                                                      | {.underline}, |
|                                                      |               |
|                                                      | Pat Hurh      |
|                                                      |               |
|                                                      | Charlie       |
|                                                      | Cooper        |
+------------------------------------------------------+---------------+
| 8.0 Accelerator Upgrades (WBS 1/2.0.1, 1/2.0.2)      | [Thomas       |
|                                                      | Roser]        |
|                                                      | {.underline}, |
|                                                      |               |
|                                                      | Rod Gerig     |
+------------------------------------------------------+---------------+
| 9.0 NuMI Beamline Upgrades (WBS 1/2.0.3, 1.0.4)      | [Phil         |
|                                                      | Martin]       |
| a\) Beamline / Target Modifications                  | {.underline}, |
|                                                      |               |
| b\) Shielding                                        | Sayed Rokni   |
+------------------------------------------------------+---------------+
| 10.0 Cost and Schedule                               | [Bill         |
|                                                      | Boroski,      |
|                                                      | ]{.underline} |
|                                                      |               |
|                                                      | Dean Hoffer   |
+------------------------------------------------------+---------------+
| 11.0 Project Management (WBS 1.9 & 2.10)             | [Mike         |
|                                                      | Lindgren]     |
|                                                      | {.underline}, |
|                                                      |               |
|                                                      | Ed Temple     |
+------------------------------------------------------+---------------+
| 12.0 Charge Questions                                |               |
+------------------------------------------------------+---------------+
| TECHNICAL                                            |               |
+------------------------------------------------------+---------------+
| 12.1 Are the technical specifications clearly stated | [Heidi        |
| and documented?                                      | Schellman,    |
|                                                      | ]{.underline} |
|                                                      |               |
|                                                      | Tom Roser     |
+------------------------------------------------------+---------------+
| 12.2 Can the design be built? Does the design meet   | [Heidi        |
| the technical specifications? Is it a reasonable     | Schellman,    |
| design?                                              | ]{.underline} |
|                                                      |               |
|                                                      | Tom Roser     |
+------------------------------------------------------+---------------+
| 12.3 Does the baseline design meet the project's     | [Heidi        |
| objectives (mission need)?                           | Schellman,    |
|                                                      | ]{.underline} |
|                                                      |               |
|                                                      | Tom Roser     |
+------------------------------------------------------+---------------+
| [COST]{.underline}                                   |               |
+------------------------------------------------------+---------------+
| 12.4 Is the Work Breakdown Structure (WBS)           | [Bill         |
| appropriate for the project scope?                   | Boroski,      |
|                                                      | ]{.underline} |
|                                                      |               |
|                                                      | Dean Hoffer   |
+------------------------------------------------------+---------------+
| 12.5 Do the cost estimates for each WBS (or cost)    |               |
| element have a sound documented basis and are they   |               |
| reasonable?                                          |               |
+------------------------------------------------------+---------------+
| 12.6 Does an obligation profile exist? How does it   |               |
| compare with the funding guidance?                   |               |
+------------------------------------------------------+---------------+
| SCHEDULE                                             |               |
+------------------------------------------------------+---------------+
| 12.7 Is the schedule well developed and              | [Dean         |
| appropriately structured by specifying               | Hoffer]       |
| relationships, predecessors, successors, critical    | {.underline}, |
| path, resource loaded, etc?                          |               |
|                                                      | Bill Boroski  |
+------------------------------------------------------+---------------+
| 12.8 Are the durations for the activities and        |               |
| overall schedule reasonable and achievable with the  |               |
| assumed resources?                                   |               |
+------------------------------------------------------+---------------+
| 12.9 Does the schedule contain appropriate levels of |               |
| milestones, sufficient quantity of milestones for    |               |
| tracking progress and do they appear to be           |               |
| achievable?                                          |               |
+------------------------------------------------------+---------------+
| 12.10 Does the schedule include activities for       |               |
| design reviews, which include assessment of the      |               |
| designs readiness for procuring prototypes,          |               |
| preproduction and production materials?              |               |
+------------------------------------------------------+---------------+
| MANAGEMENT                                           |               |
+------------------------------------------------------+---------------+
| 12.11 Is there an appropriate management             | [Mike         |
| organizational structure in place to accomplish the  | Lindgren]     |
| design and construction?                             | {.underline}, |
|                                                      |               |
|                                                      | Bill Boroski  |
+------------------------------------------------------+---------------+
| 12.12 Is the organization structure well documented, |               |
| responsibilities defined and appropriate for the     |               |
| scope of work?                                       |               |
+------------------------------------------------------+---------------+
| 12.13 Are there adequate staffing resources          |               |
| available or planned for this effort?                |               |
+------------------------------------------------------+---------------+
| 12.14 Is there a funding plan available or proposed  |               |
| to meet the resource requirements to realize the     |               |
| project?                                             |               |
+------------------------------------------------------+---------------+
| 12.15 Has a Risk Plan been developed, risks          |               |
| identified, risks analyzed, risk responses           |               |
| planned/implemented, risk monitoring/control process |               |
| established and do they seem appropriate?            |               |
+------------------------------------------------------+---------------+
| PROCUREMENT                                          |               |
+------------------------------------------------------+---------------+
| 12.16 Have the critical procurements been identified | [Joe          |
| and are they included in the schedule with adequate  | Ingraffia,    |
| lead time built in?                                  | ]{.underline} |
|                                                      |               |
|                                                      | Mike Lindgren |
+------------------------------------------------------+---------------+
| 12.17 Have critical make vs. buy decisions been      | [Joe          |
| evaluated in conjunction with the scope and is that  | Ingraffia,    |
| reflected in the baseline cost estimate, schedule    | ]{.underline} |
| and technical risk plan?                             |               |
|                                                      | Mike Lindgren |
+------------------------------------------------------+---------------+
| 12.18 Are the Project designs final and procurement  | [Joe          |
| packages prepared to the degree appropriate to order | Ingraffia,    |
| materials and initiate construction as scheduled?    | ]{.underline} |
|                                                      |               |
|                                                      | Mike Lindgren |
+------------------------------------------------------+---------------+

Note underlined names are the primary writer.

Appendix E

[]{#__RefHeading___Toc169312708 .anchor}Reviewer Assignments for
Breakout Sessions

for the Director's CD-2/3a Review of the NOvA Project

June 4 -- 6, 2007

+------------------------------------------------------+---------------+
| 1\) Site and Building (Confessional, WH5NE)          | Karen         |
|                                                      | Hellman,      |
|                                                      |               |
|                                                      | Jeff Sims     |
+------------------------------------------------------+---------------+
| 2\) Commodities -- Scintillator/Fiber/PVC (Snake Pit | Joe           |
| -- WH2NE)                                            | Ingraffia,    |
|                                                      |               |
|                                                      | Linda Stutte  |
+------------------------------------------------------+---------------+
| 3\) Far and Near Detector Assembly (The Req. Room -- | Richard       |
| WH4NW)                                               | Boyce,        |
|                                                      |               |
|                                                      | Charlie       |
|                                                      | Cooper,       |
|                                                      |               |
|                                                      | Pat Hurh      |
+------------------------------------------------------+---------------+
| 4\) Electronics and DAQ (Hornets Nest - WH8)         | Jonathan      |
|                                                      | Lewis,        |
|                                                      |               |
|                                                      | Eric James    |
+------------------------------------------------------+---------------+
| 5\) Extrusion Module Production (Black Hole --       | Alan Bross,   |
| WH2NW)                                               |               |
|                                                      | Heidi         |
|                                                      | Schellman     |
+------------------------------------------------------+---------------+
| 6\) Accelerator and NuMI Beamline Upgrades           | Rod Gerig,    |
| (Racetrack -- WH7X)                                  |               |
|                                                      | Phil Martin,  |
|                                                      |               |
|                                                      | Sayed Rokni,  |
|                                                      |               |
|                                                      | Thomas Roser  |
+------------------------------------------------------+---------------+
| 7), Cost, Schedule and Management (Comitium, WH2SE)  | Bill Boroski, |
|                                                      |               |
|                                                      | Mike          |
|                                                      | Lindgren,     |
|                                                      |               |
|                                                      | Dean Hoffer,  |
|                                                      |               |
|                                                      | Ed Temple     |
+------------------------------------------------------+---------------+

Appendix F

[]{#__RefHeading___Toc169312709 .anchor}Reviewers' Contact Information

for the Director's CD-2/3a Review of the NOvA Project

June 4 -- 6, 2007

  --------------------------------------------- -----------------------------------------
  Bill Boroski                                  Richard Boyce

  Fermi National Accelerator Laboratory         Stanford Linear Accelerator Center

  M.S. 127                                      M.S. 103

  P.O. Box 500                                  2575 Sand Hill Road

  Batavia, IL. 60510                            Menlo Park, CA 94025

  630-840-4344                                  650-926-3441

  <boroski@fnal.gov>                            <boyce@SLAC.Stanford.edu>

                                                

  Alan Bross                                    Charlie Cooper

  Fermi National Accelerator Laboratory         Fermi National Accelerator Laboratory

  M.S. 231                                      M.S. 316

  P.O. Box 500                                  P.O. Box 500

  Batavia, IL. 60510                            Batavia, IL. 60510

  630-840-4880                                  630-840-2538

  <bross@fnal.gov>                              <ccooper@fnal.gov>

                                                

  Rod Gerig                                     Karen Hellman

  Argonne National Laboratory                   Argonne National Laboratory

  9700 S. Cass Avenue                           9700 S. Cass Avenue

  Argonne, IL 60439                             Argonne, IL 60439

  630-252-5710                                  630-252-7808

  <rod@aps.anl.gov>                             [khellman@anl.gov](../khellman@anl.gov)

                                                

  Dean Hoffer                                   Pat Hurh

  Fermi National Accelerator Laboratory         Fermi National Accelerator Laboratory

  M.S. 200                                      MS 340

  P.O. Box 500                                  P.O. Box 500

  Batavia, IL. 60510                            Batavia, IL. 60510

  630-840-8898                                  630-840-2814

  <dhoffer@fnal.gov>                            <hurh@fnal.gov>

                                                

  Joe Ingraffia                                 Eric James

  Argonne National Laboratory                   Fermi National Accelerator Laboratory

  9700 S. Cass Avenue                           M.S. 318

  Argonne, IL 60439                             P.O. Box 500

  630-252-3640                                  Batavia, IL. 60510

  [jingraffia@anl.gov](../jingraffia@anl.gov)   630-840-8610

                                                <jameseb@fnal.gov>

                                                

  Jonathan Lewis                                Michael Lindgren

  Fermi National Accelerator Laboratory         Fermi National Accelerator Laboratory

  M.S. 318                                      M.S. 318

  P.O. Box 500                                  P.O. Box 500

  Batavia, IL. 60510                            Batavia, IL. 60510

  630-840-3779                                  630-840-8409

  [jdl@fnal.gov](../jdl@fnal.gov)               <mlindgre@fnal.gov>

                                                
  --------------------------------------------- -----------------------------------------

  ----------------------------------------------- -----------------------------------------------------------------
  Philip Martin                                   Sayed Rokni

  Consultant                                      Stanford Linear Accelerator Center

  623 Antler Ridge Rd                             M.S. 48

  Sequim WA 98382                                 2575 Sand Hill Road

  360-582-9445                                    Menlo Park, CA 94025

  [ptmartin@olypen.com](../ptmartin@olypen.com)   650-926-3544

                                                  <rokni@SLAC.Stanford.edu>

                                                  

  Thomas Roser                                    Heidi Schellman

  Brookhaven National Laboratory                  Northwestern University

  P.O. Box 5000                                   WCAS Physics and Astronomy EV4020

  Upton, NY 11973                                 1918 Sheridan Rd

  631-344-7084                                    Evanston, IL. 60208

  <roser@bnl.gov>                                 847-491-7561

                                                  [h-schellman@northwestern.edu](../h-schellman@northwestern.edu)

                                                  

  Jeff Sims                                       Linda Stutte

  Argonne National Laboratory                     Fermi National Accelerator Laboratory

  9700 S. Cass Avenue                             M.S. 357

  Argonne, IL 60439                               P.O. Box 500

  630-252-3515                                    Batavia, IL. 60510

  <jsims@anl.gov>                                 630-840-3108

                                                  <stutte@fnal.gov>

                                                  

  Ed Temple (Chair)                               Jeffrey Cotton (Observer)

  Fermi National Accelerator Laboratory           EG&G at Fermi National Accelerator

  M.S. 200                                        Laboratory

  P.O. Box 500                                    M.S. 210

  Batavia, IL. 60510                              P.O. Box 500

  630-840-5242                                    Batavia, IL. 60510

  <etemple@fnal.gov>                              630-840-2327

                                                  <cotton@fnal.gov>
  ----------------------------------------------- -----------------------------------------------------------------

Appendix G

[]{#__RefHeading___Toc169312710 .anchor}Participant List

for the Director's CD-2/3a Review of the NOvA Project

June 4 -- 6, 2007

  ------------------------------------------------------------------------------
  **Role**                  **Last Name**     **First     **Institution**
                                              Name**      
  ------------------------- ----------------- ----------- ----------------------
  Directorate               Appel             Jeff        Fermilab/Directorate

                            Holmes            Steve       Fermilab/Directorate

                            Montgomery        Hugh        Fermilab/Directorate

  DOE                       Procario          Mike        DOE

                            Carolan           Pepin       DOE SO

                            Webster           Steve       DOE SO

  NOvA ANU Project EE       Ducar             Robert      Fermilab

  NOvA ANU Project ME       Reilly            Robert      Fermilab

  NOvA ANU Safety           Andrews           Michael     Fermilab
  Coordinator                                             

  NOvA Executive Board      Harris            Debbie      Fermilab
  Member                                                  

  NOvA Level 3 Managers     Bernstein         Bob         Fermilab

                            Bower             Chuck       Indiana University

                            Chase             Tom         University of
                                                          Minnesota

                            Coan              Tom         Southern Methodist
                                                          Univ.

                            Cronin-Hennessy   Dan         University of
                                                          Minnesota

                            Derwent           Paul        Fermilab

                            Grozis            Chuck       Fermilab

                            Grudzinski        Jim         Argonne

                            Guarino           Vic         Argonne

                            Guglielmo         Gerry       Fermilab

                            Kephart           Karen       Fermilab

                            Kourbannis        Ioanis      Fermilab

                            Marshak           Marvin      University of
                                                          Minnesota

                            Martens           Mike        Fermilab

                            Miller            Bil         University of
                                                          Minnesota

                            Pavlicek          Vince       Fermilab

                            Peterson          Earl        University of
                                                          Minnesota

                            Plunkett          Rob         Fermilab

                            Poling            Ron         University of
                                                          Minnesota

                            Rusack            Roger       University of
                                                          Minnesota

                            Votava            Margaret    Fermilab

                            Zwaska            Robert      Fermilab

  NOvA Level 4 Managers     Ader              Christine   Fermilab

                            Broemmelsiek      Daniel      Fermilab

                            Capista           David       Fermilab

                            Childress         Sam         Fermilab

                            Dey               Joseph      Fermilab

                            Hu                Martin      Fermilab

                            Hylen             Jim         Fermilab

                            Jensen            Chris       Fermilab

                            Johnson           David       Fermilab

                            Kasper            Peter       Fermilab

                            Kobilarcik        Thomas      Fermilab

                            Tariq             Salman      Fermilab

                            Vaziri            Kamran      Fermilab

                            Williams          Karlton     Fermilab

  NOvA Project Office       Domann            Ken         Fermilab

                            Ferguson          Harry       Fermilab

                            Freeman           Bill        Fermilab

                            Grossman          Nancy       Fermilab

                            McCluskey         Elaine      Fermilab

                            Oliver            John        Harvard

                            Pasek             Suzanne     Fermilab

                            Pla-Dalmau        Anna        Fermilab

                            Pushka            David       Fermilab

                            Schuh             Keith       Fermilab

                            Wehmann           Alan        Fermilab

  NOνA Presenters           Ayres             Dave        Argonne

                            Bromberg          Carl        Michigan State
                                                          University

                            Cooper            John        Fermilab

                            Dixon             Steve       Fermilab

                            Feldman           Gary        Harvard University

                            Heller            Kenneth     University of
                                                          Minnesota

                            Messier           Mark        Indiana University

                            Mualem            Leon        Caltech

                            Mufson            Stuart      Indiana University

                            Ray               Ronald      Fermilab

                            Talaga            Richard     Argonne

  Observer                  Cotton            Jeffrey     EG&G at Fermilab

  Other Participants        Baller            Bruce       Fermilab

                            Bock              Greg        Fermilab

                            Bogert            Dixon       Fermilab

                            Cibic             Bob         Fermilab

                            Dixon             Roger       Fermilab

                            Harding           Dave        Fermilab

                            Kiemschies        Oliver      Fermilab

                            Lee               Ang         Fermilab

                            Rameika           Gina        Fermilab

                            Strait            Jim         Fermilab

                            Tinsley           Dave        Fermilab

                            Wildman           David       Fermilab

                            Xiao              Meiqin      Fermilab

  Reviewers                 Boroski           Bill        Fermilab

                            Boyce             Richard     SLAC

                            Bross             Alan        Fermilab

                            Cooper            Charlie     Fermilab

                            Gerig             Rod         Argonne

                            Hellman           Karen       Argonne

                            Hoffer            Dean        Fermilab

                            Hurh              Pat         Fermilab

                            Ingraffia         Joe         Argonne

                            James             Eric        Fermilab

                            Lewis             Jonathan    Fermilab

                            Lindgren          Mike        Fermilab

                            Martin            Phil        Consultant

                            Rokni             Sayed       SLAC

                            Roser             Thomas      Brookhaven

                            Schellman         Heidi       Northwestern

                            Sims              Jeffrey     Argonne

                            Stutte            Linda       Fermilab

                            Temple            Ed          Fermilab
  ------------------------------------------------------------------------------

Appendix H

[]{#__RefHeading___Toc169312711 .anchor}Table of Recommendations

for the Director's CD-2/3a Review of the NOvA Project

June 4 -- 6, 2007

+---+------------------------------------+-------+----------+---------+
| * | **Recommendation**                 | **Ass | **S      | *       |
| * |                                    | igned | tatus/** | *Date** |
| \ |                                    | To**  |          |         |
| # |                                    |       | **       |         |
| * |                                    |       | Action** |         |
| * |                                    |       |          |         |
+===+====================================+=======+==========+=========+
|   | 2.0 Science                        |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 1 | The mass and flux estimates in the |       |          |         |
|   | TDR need to be made consistent.    |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 2 | **The continued shrinkage of the   |       |          |         |
|   | NOvA far detector is an area of    |       |          |         |
|   | concern**. The collaboration needs |       |          |         |
|   | to come up with a consistent plan  |       |          |         |
|   | which allows the project to        |       |          |         |
|   | reconcile the cost guidance with   |       |          |         |
|   | maximal physics sensitivity.       |       |          |         |
+---+------------------------------------+-------+----------+---------+
|   | 3.0 Site and Building (WBS 1/2.1)  |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 3 | It is recommended that the project |       |          |         |
|   | develop a responsibility matrix    |       |          |         |
|   | that exhibits the various          |       |          |         |
|   | requirements and responsibilities  |       |          |         |
|   | of the individual team members     |       |          |         |
|   | related to the construction and    |       |          |         |
|   | oversight of the road, site and    |       |          |         |
|   | facility.                          |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 4 | It is important to further develop |       |          |         |
|   | the schedule incorporating the     |       |          |         |
|   | activities leading to an approval  |       |          |         |
|   | of CD-3a.                          |       |          |         |
+---+------------------------------------+-------+----------+---------+
|   | 4.0 Commodities --                 |       |          |         |
|   | Scintillator/Fiber/PVC (WBS 1/2.2, |       |          |         |
|   | 1/2.3 & 1/2.4)                     |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 5 | Scrub the costs and schedules in   |       |          |         |
|   | Open Plan.                         |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 6 | Develop a standard procurement     |       |          |         |
|   | milestone plan to use across the   |       |          |         |
|   | commodities.                       |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 7 | Use the current pricing estimate   |       |          |         |
|   | available for the mineral oil and  |       |          |         |
|   | recalculate the contingency.       |       |          |         |
+---+------------------------------------+-------+----------+---------+
|   | 5.0 Extrusion Module Production    |       |          |         |
|   | (WBS1/2..5)                        |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 8 | Fully automate glue application to |       |          |         |
|   | the end seal extrusion             |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 9 | Reconsider the location of Factory |       |          |         |
|   | 1 in the Fermilab Wide-Band Hall.  |       |          |         |
|   | Consider leasing a facility with   |       |          |         |
|   | enough space to incorporate both   |       |          |         |
|   | Factory 1 production activities    |       |          |         |
|   | and the needs for interim storage. |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 1 | Consider improving the fixturing   |       |          |         |
| 0 | for cutting the 32 cell assemblies |       |          |         |
|   | to length. Even with an edge       |       |          |         |
|   | guide, cutting by hand with a      |       |          |         |
|   | circular saw can still lead to an  |       |          |         |
|   | irregular edge. This could lead to |       |          |         |
|   | sealing problems. The current      |       |          |         |
|   | procedure does raise some safety   |       |          |         |
|   | concerns.                          |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 1 | Prototype and test as soon as      |       |          |         |
| 1 | possible the baseline method       |       |          |         |
|   | (packed desiccant) for insuring    |       |          |         |
|   | that the sealed gas volume         |       |          |         |
|   | surrounding the APD (interface     |       |          |         |
|   | region between the APD module and  |       |          |         |
|   | the PVC module optical connector)  |       |          |         |
|   | remains dry and prevents any       |       |          |         |
|   | possibility of condensation on the |       |          |         |
|   | APD or fiber surfaces. Some        |       |          |         |
|   | thought should be given as to how  |       |          |         |
|   | a dry N~2~ purge could be added if |       |          |         |
|   | the desiccant concept does not     |       |          |         |
|   | work well enough.                  |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 1 | The team should make use of the    |       |          |         |
| 2 | evolving 3D model of the detector  |       |          |         |
|   | to better evaluate if there will   |       |          |         |
|   | be an interference between         |       |          |         |
|   | vertical and horizontal components |       |          |         |
|   | in some parts of the detector.     |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 1 | We recommend that a purchasing     |       |          |         |
| 3 | expediter be added to the NOvA     |       |          |         |
|   | Project Office Staff at the        |       |          |         |
|   | appropriate time.                  |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 1 | Increase contingency on labor for  |       |          |         |
| 4 | Factory 1 to 50%                   |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 1 | Given the potential problems that  |       |          |         |
| 5 | might occur if the scintillator    |       |          |         |
|   | comes in contact with the Devcon   |       |          |         |
|   | PlasticWelder, we recommend that   |       |          |         |
|   | the team perform additional tests  |       |          |         |
|   | (hydrostatic) on the barrier seal  |       |          |         |
|   | to get an estimate on the expected |       |          |         |
|   | volume of scintillator that might  |       |          |         |
|   | come in contact with the Devcon    |       |          |         |
|   | adhesive.                          |       |          |         |
+---+------------------------------------+-------+----------+---------+
|   | 6.0 Electronics, Trigger DAQ (WBS  |       |          |         |
|   | 1/2..6 &1/2..7)                    |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 1 | Complete and document a full risk  |       |          |         |
| 6 | analysis for the APDs. The risk    |       |          |         |
|   | analysis should include the        |       |          |         |
|   | possible mitigations for higher    |       |          |         |
|   | dark current such as lower         |       |          |         |
|   | operating temperature or changes   |       |          |         |
|   | to FEB parameters to enable NOvA   |       |          |         |
|   | to achieve the 10:1 noise          |       |          |         |
|   | specification.                     |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 1 | A risk analysis is needed for the  |       |          |         |
| 7 | FEB and DCM. Because the           |       |          |         |
|   | production is scheduled to be late |       |          |         |
|   | in the project, parts may become   |       |          |         |
|   | obsolete. It is necessary to       |       |          |         |
|   | consider the relative merits of    |       |          |         |
|   | purchasing components early        |       |          |         |
|   | against the possibility of needing |       |          |         |
|   | to redesign the boards.            |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 1 | A system integration test          |       |          |         |
| 8 | including fully functioning APD,   |       |          |         |
|   | TEC and FEB is necessary to        |       |          |         |
|   | demonstrate the performance of the |       |          |         |
|   | APD prior to making the purchase   |       |          |         |
|   | of the requested CD-3a items.      |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 1 | The WBS needs scrubbed to reflect  |       |          |         |
| 9 | the updated plan for APD assembly: |       |          |         |
|   |                                    |       |          |         |
|   | -   Remove references to carrier   |       |          |         |
|   |     board manufacture              |       |          |         |
|   |                                    |       |          |         |
|   | -   Unify nomenclature for         |       |          |         |
|   |     "housing", "module", etc.      |       |          |         |
|   |                                    |       |          |         |
|   | -   Include module assembly either |       |          |         |
|   |     as a separate step or with     |       |          |         |
|   |     testing at CalTech and         |       |          |         |
|   |     Minnesota.                     |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 2 | Include reviews prior to purchase  |       |          |         |
| 0 | of production quantities. The      |       |          |         |
|   | approval of the designs and quote  |       |          |         |
|   | packages should be milestones for  |       |          |         |
|   | the level 3 subprojects.           |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 2 | The WBS should be scrubbed to have |       |          |         |
| 1 | proper dependencies. For example,  |       |          |         |
|   | the vertical slice tests should    |       |          |         |
|   | depend on completion of testing of |       |          |         |
|   | included devices with a review to  |       |          |         |
|   | follow.                            |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 2 | The acquisition schedule for the   |       |          |         |
| 2 | APDs needs to be clarified. The    |       |          |         |
|   | vertical slice test was reported   |       |          |         |
|   | to be delayed. The schedule shows  |       |          |         |
|   | the APD pilot production for use   |       |          |         |
|   | on the IPND to be occurring 8/07   |       |          |         |
|   | through 1/08. Given the            |       |          |         |
|   | uncertainty in APD qualification   |       |          |         |
|   | and the negotiations with          |       |          |         |
|   | Hamamatsu, there is some danger    |       |          |         |
|   | that the APDs become a critical    |       |          |         |
|   | path item for an IPND run          |       |          |         |
|   | beginning in 10/08.                |       |          |         |
+---+------------------------------------+-------+----------+---------+
|   | 7.0 Far and Near Detector Assembly |       |          |         |
|   | (WBS1/2..8 & 1/2.9)                |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 2 | Perform a careful and detailed     |       |          |         |
| 3 | study of the detector assembly     |       |          |         |
|   | program with consideration given   |       |          |         |
|   | to programmatic impacts from       |       |          |         |
|   | normal equipment maintenance and   |       |          |         |
|   | worker safety resulting from the   |       |          |         |
|   | project's ISM plan. Included in    |       |          |         |
|   | this study should be a review of   |       |          |         |
|   | schedule impact due to equipment   |       |          |         |
|   | failure/repair as this will cause  |       |          |         |
|   | an alteration to the work planning |       |          |         |
|   | process. Recommend increasing the  |       |          |         |
|   | number of assembly staff at the    |       |          |         |
|   | Far Detector area to allow for on  |       |          |         |
|   | going worker training while        |       |          |         |
|   | maintaining a full production      |       |          |         |
|   | capability. This can be            |       |          |         |
|   | accomplished without cost increase |       |          |         |
|   | by reduction of the 100% labor     |       |          |         |
|   | contingency.                       |       |          |         |
+---+------------------------------------+-------+----------+---------+
|   | Near Detector& IPND Assembly       |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 2 | Reexamine the scintillator filling |       |          |         |
| 4 | equipment and excavation time      |       |          |         |
|   | estimates.                         |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 2 | Include more information in the    |       |          |         |
| 5 | BoE quotes including specific      |       |          |         |
|   | references.                        |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 2 | "Scrub" the M&S and Labor          |       |          |         |
| 6 | contingency estimates.             |       |          |         |
+---+------------------------------------+-------+----------+---------+
|   | Far Detector Assembly              |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 2 | Set the PVC formulation as soon as |       |          |         |
| 7 | possible if it is not already set  |       |          |         |
|   | at NOvA formulation 27.            |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 2 | Risk assessment needs to be done   |       |          |         |
| 8 | on all major unit operations.      |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 2 | Implement a system that triggers   |       |          |         |
| 9 | the level of review needed for     |       |          |         |
|   | each unit operation and start on   |       |          |         |
|   | major items listed in comments     |       |          |         |
|   | above.                             |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 3 | The content in the BoE's needs to  |       |          |         |
| 0 | be improved to include the sources |       |          |         |
|   | that the financial estimates come  |       |          |         |
|   | from and match them to WBS.        |       |          |         |
+---+------------------------------------+-------+----------+---------+
|   | Far Detector Structure             |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 3 | Continue with current course of    |       |          |         |
| 1 | FEA analysis and prototype/creep   |       |          |         |
|   | testing to explore various loading |       |          |         |
|   | conditions, sensitivities, and     |       |          |         |
|   | verification of analysis. This     |       |          |         |
|   | includes the close to full height  |       |          |         |
|   | 2 block test with water. The use   |       |          |         |
|   | of water is deemed satisfactory    |       |          |         |
|   | for these structural tests.        |       |          |         |
|   | Structural creep tests require     |       |          |         |
|   | long periods of data collection    |       |          |         |
|   | and the delay of starting such a   |       |          |         |
|   | test from contending with the ES&H |       |          |         |
|   | and handling issues of large       |       |          |         |
|   | quantities of liquid scintillator  |       |          |         |
|   | should be avoided. Chemical        |       |          |         |
|   | interactions between PVC and the   |       |          |         |
|   | scintillator oil ingredients can   |       |          |         |
|   | be done in smaller stand-alone     |       |          |         |
|   | tests. If interactions are noted,  |       |          |         |
|   | then long term tests can be        |       |          |         |
|   | planned and performed at that      |       |          |         |
|   | time.                              |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 3 | Investigate ways to accelerate     |       |          |         |
| 2 | block-to-block interaction tests   |       |          |         |
|   | that depend on creep and/or creep  |       |          |         |
|   | induced buckling (elevated         |       |          |         |
|   | temperature).                      |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 3 | Plan a rigorous technical review   |       |          |         |
| 3 | (or series of reviews) of the      |       |          |         |
|   | base-line structure design         |       |          |         |
|   | utilizing external experts as soon |       |          |         |
|   | as reasonably possible. This       |       |          |         |
|   | should include experts in the      |       |          |         |
|   | fields of plastic creep behavior,  |       |          |         |
|   | non-linear FEA, and buckling.      |       |          |         |
+---+------------------------------------+-------+----------+---------+
|   | Far Detector Assembly Equipment    |       |          |         |
|   | (incl. scintillator filling)       |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 3 | Continue to advance design and     |       |          |         |
| 4 | prototyping of key assembly        |       |          |         |
|   | equipment such that ergonomic and  |       |          |         |
|   | ES&H concerns can be addressed     |       |          |         |
|   | early in the design cycle.         |       |          |         |
+---+------------------------------------+-------+----------+---------+
|   | Cost, Schedule, Resources, & Risk  |       |          |         |
|   | Management - Far Detector          |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 3 | Provide tie-ins between BoE and    |       |          |         |
| 5 | technical documents.               |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 3 | Increase resolution of BoE's (by   |       |          |         |
| 6 | writing more BoE's at a lower      |       |          |         |
|   | level or showing breakdown of      |       |          |         |
|   | estimates within those BoE's       |       |          |         |
|   | written to cover multiple WBS      |       |          |         |
|   | activities; in the latter case,    |       |          |         |
|   | clearly indicate on the multiple   |       |          |         |
|   | activity BoE's which activities or |       |          |         |
|   | portions of activities are         |       |          |         |
|   | included).                         |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 3 | Continue with planned assembly     |       |          |         |
| 7 | tests and studies (time-motion) to |       |          |         |
|   | provide basis of estimate for      |       |          |         |
|   | schedules.                         |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 3 | Perform risk analysis and          |       |          |         |
| 8 | "what-if" analysis to assess       |       |          |         |
|   | schedule and cost impact from      |       |          |         |
|   | downtime of key assembly equipment |       |          |         |
|   | (glue dispenser, vacuum lifters,   |       |          |         |
|   | cranes, block pivoter,             |       |          |         |
|   | scintillation filling equipment,   |       |          |         |
|   | etc.)                              |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 3 | Scrub the BoE's and RLS to attain  |       |          |         |
| 9 | consistency.                       |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 4 | Write BoE's for R&D activities.    |       |          |         |
| 0 |                                    |       |          |         |
+---+------------------------------------+-------+----------+---------+
|   | 8.0 Accelerator Upgrades (WBS      |       |          |         |
|   | 1/2..0.1 & 1/2.0.2                 |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 4 | Include complete system            |       |          |         |
| 1 | integration testing in the scope   |       |          |         |
|   | of the project.                    |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 4 | Complete the conceptual design as  |       |          |         |
| 2 | soon as possible and then review   |       |          |         |
|   | the contingencies used in the      |       |          |         |
|   | engineering designs.               |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 4 | Consider installing the prototype  |       |          |         |
| 3 | gap cleaner or the final magnet    |       |          |         |
|   | built early in the MI for early    |       |          |         |
|   | testing and use by NuMI. The gap   |       |          |         |
|   | cleaner can later be moved to RR.  |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 4 | Consider minimizing beam losses in |       |          |         |
| 4 | the Recycler as part of the design |       |          |         |
|   | efforts.                           |       |          |         |
+---+------------------------------------+-------+----------+---------+
|   | 9.0 NuMI Beamline Upgrades (WBS    |       |          |         |
|   | 1/2..0)                            |       |          |         |
+---+------------------------------------+-------+----------+---------+
|   | 9.1 Beamline/Target Modifications  |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 4 | Pursue the use of continuous cast  |       |          |         |
| 5 | salvage steel for the shielding    |       |          |         |
|   | reconfiguration. Also see if some  |       |          |         |
|   | or all of the steel might be       |       |          |         |
|   | available on site at minimal cost. |       |          |         |
+---+------------------------------------+-------+----------+---------+
|   | 9.2 Shielding                      |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 4 | Prepare a status report on the     |       |          |         |
| 6 | issue of tritium in the surface    |       |          |         |
|   | water, and discuss why this issue  |       |          |         |
|   | does not pose a risk to the        |       |          |         |
|   | project when beam power to NuMI is |       |          |         |
|   | increased.                         |       |          |         |
+---+------------------------------------+-------+----------+---------+
|   | 10.0 Cost                          |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 4 | Complete a thorough and rigorous   |       |          |         |
| 7 | review of the newly revised cost   |       |          |         |
|   | estimate to verify the accuracy    |       |          |         |
|   | and completeness of the June 2007  |       |          |         |
|   | estimate prior to baseline         |       |          |         |
|   | consideration.                     |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 4 | Review the cost basis for each     |       |          |         |
| 8 | element in the WBS to ensure       |       |          |         |
|   | that 1) BoE documents are          |       |          |         |
|   | correctly matched with WBS line    |       |          |         |
|   | items; and 2) each BoE contains    |       |          |         |
|   | the appropriate and necessary      |       |          |         |
|   | information to support the cost    |       |          |         |
|   | estimate.                          |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 4 | Verify that peer and independent   |       |          |         |
| 9 | design reviews are incorporated    |       |          |         |
|   | into the WBS at the appropriate    |       |          |         |
|   | level and properly costed.         |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 5 | Review the level of contingency    |       |          |         |
| 0 | assigned to WBS line items by      |       |          |         |
|   | Level 2 managers to ensure that    |       |          |         |
|   | labor, M&S and schedule            |       |          |         |
|   | contingency rules are being        |       |          |         |
|   | applied uniformly across the       |       |          |         |
|   | project.                           |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 5 | Update the price projection model  |       |          |         |
| 1 | for mineral oil, using May 2007    |       |          |         |
|   | vendor cost data of \$3.33/gallon. |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 5 | Perform and document a             |       |          |         |
| 2 | cost-benefit analysis to determine |       |          |         |
|   | the potential cost savings of      |       |          |         |
|   | combining PVC module assembly      |       |          |         |
|   | production into a single factory   |       |          |         |
|   | site.                              |       |          |         |
+---+------------------------------------+-------+----------+---------+
|   | 11.0 Schedule (To be completed     |       |          |         |
|   | prior to DOE CD-2/3a Review)       |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 5 | The schedule is to be scrubbed to  |       |          |         |
| 3 | reduce cost, including modifying   |       |          |         |
|   | the detector size, in order to     |       |          |         |
|   | meet the OMB guidance not to       |       |          |         |
|   | exceed a TPC of \$260M.            |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 5 | To insure schedule quality, the    |       |          |         |
| 4 | schedule is to be scrubbed for the |       |          |         |
|   | items listed in the first Comment  |       |          |         |
|   | of the schedule section (11) of    |       |          |         |
|   | this report.                       |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 5 | Complete the detailed bottom-up    |       |          |         |
| 5 | risk assessment for the entire     |       |          |         |
|   | project, which includes mitigation |       |          |         |
|   | plans. Incorporate the mitigation  |       |          |         |
|   | activities in the cost and         |       |          |         |
|   | schedule where appropriate.        |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 5 | Assess at which level(s) in the    |       |          |         |
| 6 | WBS the BoEs should be generated.  |       |          |         |
|   | Complete all the BoE's for the     |       |          |         |
|   | project. Compare the content of    |       |          |         |
|   | the BoEs with the data in the      |       |          |         |
|   | schedule and insure they match     |       |          |         |
|   | (Similar Recommendation issued     |       |          |         |
|   | from the Director's CD-1 Review of |       |          |         |
|   | NOvA).                             |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 5 | Generate and maintain a one-page   |       |          |         |
| 7 | master schedule with the critical  |       |          |         |
|   | path.                              |       |          |         |
+---+------------------------------------+-------+----------+---------+
|   | 12.0 Project Management (WBS       |       |          |         |
|   | 1/2..0.1 & 1/2.0.2                 |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 5 | It is important to have a draft    |       |          |         |
| 8 | MOU begun between Fermilab and the |       |          |         |
|   | University of Minnesota to         |       |          |         |
|   | supplement the CA.                 |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 5 | Prepare draft MOU's and SOW's for  |       |          |         |
| 9 | institutions planning on doing     |       |          |         |
|   | work for the project so that       |       |          |         |
|   | responsibilities and labor         |       |          |         |
|   | resources commitments are clearly  |       |          |         |
|   | understood.                        |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 6 | Finalize both PEP and PMP prior to |       |          |         |
| 0 | a baseline review.                 |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 6 | There needs to be detailed balance |       |          |         |
| 1 | of funding and cost by FY by the   |       |          |         |
|   | baseline review in the project     |       |          |         |
|   | management documentation.          |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 6 | The project office needs to work   |       |          |         |
| 2 | with the L2 managers to complete   |       |          |         |
|   | the implementation of the Risk     |       |          |         |
|   | Management Plan.                   |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 6 | Complete the TDR and scrub all     |       |          |         |
| 3 | CD-2 related documentation to      |       |          |         |
|   | reflect the current project status |       |          |         |
|   | and scope.                         |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 6 | Consider adding an expediter to    |       |          |         |
| 4 | the Project office.                |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 6 | Include costs in WBS 2.10 for      |       |          |         |
| 5 | external reviewers/consultants,    |       |          |         |
|   | labor contingency, and an          |       |          |         |
|   | expediter.                         |       |          |         |
+---+------------------------------------+-------+----------+---------+
| 6 | Preparation of materials for the   |       |          |         |
| 6 | baseline review should begin early |       |          |         |
|   | enough that the reviewers have     |       |          |         |
|   | adequate time to prepare for the   |       |          |         |
|   | review.                            |       |          |         |
+---+------------------------------------+-------+----------+---------+