----------------------------------------------------- ----------------------------------------------
  ![](media/image1.jpeg){width="2.1041666666666665in"   Fiscal Year 2003
  height="0.6263888888888889in"}                        

                                                        Annual Performance Evaluation and Appraisal

                                                        

  Lawrence Livermore National Laboratory                
  ----------------------------------------------------- ----------------------------------------------

![](media/image2.wmf){width="4.554861111111111in"
height="2.0381944444444446in"}

+-----------------------------------------------------------------------+
| **Prepared by:**                                                      |
|                                                                       |
| **Livermore Site Office**                                             |
|                                                                       |
| **National Nuclear Security Administration**                          |
|                                                                       |
| **December 2003**                                                     |
+-----------------------------------------------------------------------+

**CONTRACTING OFFICER'S EVALUATION**

The NNSA Livermore Site Office Manager reviewed and discussed the
recommendations of functional managers and staff concerning the
appropriate adjectival ratings with which to rate the University of
California's performance in the management and operation of the Lawrence
Livermore National Laboratory. Based upon this process, an adjectival
rating of "**Outstanding**" is earned for Mission, and a
**"Satisfactory"** is earned for Operations. This report, the "Fiscal
Year 2003 Annual Performance Evaluation and Appraisal - Lawrence
Livermore National Laboratory" provides the basis for my determination,
and is hereby endorsed and approved.

![](media/image3.jpeg){width="6.498611111111111in"
height="1.742361111111111in"}

FY 2003 Annual Performance Evaluation and Appraisal

for

Lawrence Livermore National Laboratory

**Table of Contents**

+-----+-----+-----+---+--------------------------------------+------+
| *   |     |     |   |                                      | *    |
| *In |     |     |   |                                      | *1** |
| tro |     |     |   |                                      |      |
| duc |     |     |   |                                      |      |
| tio |     |     |   |                                      |      |
| n** |     |     |   |                                      |      |
+-----+-----+-----+---+--------------------------------------+------+
|     | **E |     |   |                                      |      |
|     | val |     |   |                                      |      |
|     | uat |     |   |                                      |      |
|     | ion |     |   |                                      |      |
|     | Pro |     |   |                                      |      |
|     | ces |     |   |                                      |      |
|     | s** |     |   |                                      |      |
+-----+-----+-----+---+--------------------------------------+------+
|     | *   |     |   |                                      |      |
|     | *Pe |     |   |                                      |      |
|     | rfo |     |   |                                      |      |
|     | rma |     |   |                                      |      |
|     | nce |     |   |                                      |      |
|     | Pe  |     |   |                                      |      |
|     | rio |     |   |                                      |      |
|     | d** |     |   |                                      |      |
+-----+-----+-----+---+--------------------------------------+------+
|     |     |     |   |                                      |      |
+-----+-----+-----+---+--------------------------------------+------+
| **O |     |     |   |                                      | *    |
| ver |     |     |   |                                      | *3** |
| all |     |     |   |                                      |      |
| App |     |     |   |                                      |      |
| rai |     |     |   |                                      |      |
| sal |     |     |   |                                      |      |
| Res |     |     |   |                                      |      |
| ult |     |     |   |                                      |      |
| s** |     |     |   |                                      |      |
+-----+-----+-----+---+--------------------------------------+------+
|     |     |     |   |                                      |      |
+-----+-----+-----+---+--------------------------------------+------+
| *   |     |     |   |                                      | **   |
| *De |     |     |   |                                      | 10** |
| tai |     |     |   |                                      |      |
| led |     |     |   |                                      |      |
| App |     |     |   |                                      |      |
| rai |     |     |   |                                      |      |
| sal |     |     |   |                                      |      |
| Res |     |     |   |                                      |      |
| ult |     |     |   |                                      |      |
| s** |     |     |   |                                      |      |
+-----+-----+-----+---+--------------------------------------+------+
|     |     |     |   |                                      |      |
+-----+-----+-----+---+--------------------------------------+------+
|     | **  |     |   |                                      | **   |
|     | Mis |     |   |                                      | 10** |
|     | sio |     |   |                                      |      |
|     | n** |     |   |                                      |      |
+-----+-----+-----+---+--------------------------------------+------+
|     | >   |     |   |                                      | **   |
|     |  ** |     |   |                                      | 10** |
|     | Obj |     |   |                                      |      |
|     | ect |     |   |                                      |      |
|     | ive |     |   |                                      |      |
|     | >   |     |   |                                      |      |
|     | 1** |     |   |                                      |      |
+-----+-----+-----+---+--------------------------------------+------+
|     | >   |     |   |                                      | **   |
|     |  ** |     |   |                                      | 12** |
|     | Obj |     |   |                                      |      |
|     | ect |     |   |                                      |      |
|     | ive |     |   |                                      |      |
|     | >   |     |   |                                      |      |
|     | 2** |     |   |                                      |      |
+-----+-----+-----+---+--------------------------------------+------+
|     | >   |     |   |                                      | **   |
|     |  ** |     |   |                                      | 15** |
|     | Obj |     |   |                                      |      |
|     | ect |     |   |                                      |      |
|     | ive |     |   |                                      |      |
|     | >   |     |   |                                      |      |
|     | 3** |     |   |                                      |      |
+-----+-----+-----+---+--------------------------------------+------+
|     | >   |     |   |                                      | **   |
|     |  ** |     |   |                                      | 18** |
|     | Obj |     |   |                                      |      |
|     | ect |     |   |                                      |      |
|     | ive |     |   |                                      |      |
|     | >   |     |   |                                      |      |
|     | 4** |     |   |                                      |      |
+-----+-----+-----+---+--------------------------------------+------+
|     | >   |     |   |                                      | **   |
|     |  ** |     |   |                                      | 25** |
|     | Obj |     |   |                                      |      |
|     | ect |     |   |                                      |      |
|     | ive |     |   |                                      |      |
|     | >   |     |   |                                      |      |
|     | 5** |     |   |                                      |      |
+-----+-----+-----+---+--------------------------------------+------+
|     | >   |     |   |                                      | **   |
|     |  ** |     |   |                                      | 43** |
|     | Obj |     |   |                                      |      |
|     | ect |     |   |                                      |      |
|     | ive |     |   |                                      |      |
|     | >   |     |   |                                      |      |
|     | 6** |     |   |                                      |      |
+-----+-----+-----+---+--------------------------------------+------+
|     |     |     |   |                                      |      |
+-----+-----+-----+---+--------------------------------------+------+
|     | **  |     |   |                                      | **   |
|     | Ope |     |   |                                      | 48** |
|     | rat |     |   |                                      |      |
|     | ion |     |   |                                      |      |
|     | s** |     |   |                                      |      |
+-----+-----+-----+---+--------------------------------------+------+
|     | >   |     |   |                                      | **   |
|     |  ** |     |   |                                      | 48** |
|     | Obj |     |   |                                      |      |
|     | ect |     |   |                                      |      |
|     | ive |     |   |                                      |      |
|     | >   |     |   |                                      |      |
|     | 7** |     |   |                                      |      |
+-----+-----+-----+---+--------------------------------------+------+
|     | >   |     |   |                                      | **   |
|     |  ** |     |   |                                      | 60** |
|     | Obj |     |   |                                      |      |
|     | ect |     |   |                                      |      |
|     | ive |     |   |                                      |      |
|     | >   |     |   |                                      |      |
|     | 8** |     |   |                                      |      |
+-----+-----+-----+---+--------------------------------------+------+
|     | >   |     |   |                                      | **   |
|     |  ** |     |   |                                      | 65** |
|     | Obj |     |   |                                      |      |
|     | ect |     |   |                                      |      |
|     | ive |     |   |                                      |      |
|     | >   |     |   |                                      |      |
|     | 9** |     |   |                                      |      |
+-----+-----+-----+---+--------------------------------------+------+
|     |     |     |   |                                      |      |
+-----+-----+-----+---+--------------------------------------+------+
| **  |     |     |   |                                      | **   |
| App |     |     |   |                                      | 66** |
| end |     |     |   |                                      |      |
| ice |     |     |   |                                      |      |
| s** |     |     |   |                                      |      |
+-----+-----+-----+---+--------------------------------------+------+
|     | *   |     |   | **Ratings**                          | **   |
|     | *Ap |     |   |                                      | 66** |
|     | pen |     |   |                                      |      |
|     | dix |     |   |                                      |      |
|     | A** |     |   |                                      |      |
+-----+-----+-----+---+--------------------------------------+------+
|     | *   |     |   | **Acronyms**                         | **   |
|     | *Ap |     |   |                                      | 70** |
|     | pen |     |   |                                      |      |
|     | dix |     |   |                                      |      |
|     | B** |     |   |                                      |      |
+-----+-----+-----+---+--------------------------------------+------+

**Introduction**

This report was produced by the U. S. Department of Energy (DOE)
National Nuclear Security Administration (NNSA), Livermore Site Office
(LSO) to provide the Contracting Officer's written assessment of the
Contractor's performance at the Lawrence Livermore National Laboratory
(LLNL) under contract W-7405-ENG-48, Appendix F. Contract Appendix F
defines the Objective Standards of Performance agreed to by DOE/NNSA and
the University of California (Contractor or UC) to annually measure the
Contractor's overall performance of its Science and Technology (S&T)
Mission and its Operations. UC is eligible to earn program performance
fee based on the Objective Standards of Performance listed in Appendix F
of the contracts.

There may be programs, systems, compliance requirements or observations
not covered by

Appendix F presented in this report. These additional observations are
limited to items of performance that require the attention of the
Laboratory Director, but are not effectively covered by Appendix F
performance measures. Although these items are included in this report,
they do not contribute to the basis for the overall rating of Contractor
performance under Appendix F.

**Evaluation Process**

The Contractor and NNSA have agreed to use a performance-based
management system for Laboratory oversight as part of the contract.
These standards are used for the appraisal and evaluation of work under
this contract. The primary objective of this report is to provide a
summary of the annual Contracting Officer's written assessment of the
Contractor's performance and the amount of earned Program Performance
Fee as specified in contract clause H.007 and H.014, respectively. The
parties agree that the purpose of the Appendix F is to focus on
strategic and mission-critical activities (i.e., the "critical few") and
to appraise the Contractor's systems and outcomes in terms of:

-   Are they producing appropriate national security, science and
    technology results? and

-   Are they producing these results efficiently, safely and securely?

The Contractor will provide an annual Contractor's Evaluation Report
assessing their performance. An annual Performance Evaluation Report
prepared by the Site Office Manager will provide an evaluation of the
Contractor's performance during the Appendix F appraisal period.
DOE/NNSA will use the Contractor's Evaluation Report as the primary
basis for the annual appraisal of Contractor performance, recognizing
that DOE/NNSA will take into account other pertinent information,
including that performance against each Strategic Performance Objective
is subject to timely availability of adequate funding, as well as
operational oversight, internal and external program reviews and audits,
consistent with the intent of this Contract, in determining the annual
appraisal for performance.

The validation effort is conducted by teams responsible for the various
Performance Objectives and Measures represented in Appendix F. These
teams, with guidance from LSO management, are responsible for developing
an adequate, independent basis for assessing the quality, credibility,
and accuracy of the Contractor\'s self-assessment. These evaluations are
used as a basis for the Contracting Officer's evaluation of the
Contractor\'s performance.

**Performance Period**

Designed to capture performance for Fiscal Year 2003, the
self-assessment period for the Laboratory is October 1, 2002 through
September 30, 2003, unless specified in the Performance Objective.
Significant performance between the later date and the end of the Fiscal
Year is to be assessed by the Laboratory and provided as a supplement to
the self-assessment. The Contractor provided the self-assessment of
LLNL, supplemental information and proposed rating to LSO in October,
2003.

**Overall Appraisal Results**

This is the first annual contract performance assessment under the
restructured Appendix F process. The Livermore Site Office (LSO) has
worked closely with NNSA HQ, the Los Alamos Site Office, the Lawrence
Livermore and Los Alamos National Laboratories and the University of
California, Office of the President, to develop, negotiate and implement
what we believe to be an improved contract assessment tool that focuses
on completing the NNSA mission as defined in the NNSA Strategic Plan
while allowing the contractor flexibility in determining how the work
will be accomplished.

In assessing the performance of Lawrence Livermore National Laboratory
(LLNL), the LSO considered, but was not limited to LLNL's
self-assessment, LSO reviews, external reviews and audits, NNSA HQ input
and LSO's daily operational oversight. Based upon these activities, LLNL
has earned an **Outstanding** rating in Mission and a **Satisfactory**
rating in Operations.

These ratings are supported by the following examples with the detailed
LSO rating sheets attached.

**[MISSION]{.underline}**

LLNL continued to effectively execute an essential role in the US
Stockpile Stewardship Program. As one of the two "physics design" labs,
LLNL is responsible for the annual assessment of the safety and
reliability of the LLNL-designed portion of the U.S. nuclear weapons
stockpile, and functions in a "peer review" mode for the rest of the
stockpile. LLNL accomplished this mission through a coupled set of
experiments, computer simulations, and theoretical analysis that
provides the data and understanding used by experienced weapons
scientists to assess specific and general weapon safety and reliability
issues. These activities balanced the short-term needs of the stockpile
with the need to develop the tools that can be used to maintain
confidence in the stockpile for the indefinite future without the use of
underground nuclear tests.

Particular areas of emphasis at LLNL during FY 2003 included:
design/planning of the refurbishment for the W80-1 warhead, support for
the final (manufacturing) phase of the refurbishment of the W87 warhead,
development (with Los Alamos National Laboratory (LANL)) of a
certification methodology based on the Quantification of Margins and
Uncertainties (QMU), execution of a coordinated set of large and
small-scale experiments supporting all aspects of the weapons program,
and development and application of new computational techniques as well
as development and operation of the advanced computers that run them.

Overall, LLNL performance on activities in support of the nuclear
weapons program was outstanding, although several areas of improvement
are noted in the specific performance measure discussions that follow.

The Counterintelligence (CI) and Security Awareness for Employees (SAFE)
programs did an excellent job in developing and implementing
nonproliferation/counterterrorism programs. Analytical projects were
conducted and assessments were provided in support of CI officers.
Several of these led to CI investigations and provided LLNL management
with timely decision making information related to risk management.

Efforts to enhance and nurture a strong science base are evidenced by
the Laboratory's outstanding performance in developing and integrating a
balanced strategy for investing programmatic and institutional resources
to ensure the long-term vitality of its science and technology base
while supporting the DOE/NNSA mission, Department of Homeland Security
(DHS) mission, and other emerging national requirements. During FY 2003,
the Laboratory Director led the effort, along with his associate
directors and a group of scientific and technical leaders, to develop a
long range scientific and technology investment plan for the next ten
years. This will enable LLNL to balance their investments in workforce,
foster interactions across disciplines, organizations, and facilities,
and position the Laboratory to continue to support the DOE/NNSA mission
as well other federal agencies. LLNL maintained an outstanding level of
science and mission relevance based on the projects validated during
this performance period.

In addition, LLNL has maintained their signature capabilities through
facility investments such as supercomputing, the laser complex,
forensics and nano sciences, and biosecurity. It has also pursued and
executed a successful portfolio on non-DOE sponsored research that
enhances LLNL's competencies to meet current and future national needs.
LLNL actively participated in the broad scientific community through
collaborations with other Laboratories, industry, and academia. Evidence
of their success includes their industrial partners success in bringing
products to the commercial marketplace and their receipt of six 2003 R&D
100 Awards marking the second consecutive year they have received six of
these awards.

Significant construction projects were executed within budget, scope and
schedule as evidenced by their successes with the Engineering Technology
Complex Upgrade (ETCU), the Sensitive Compartmented Information Facility
(SCIF) and the Terascale Simulation Facility (TSF). TSF is likely to
come in well ahead of schedule; ETCU had an identified need to increase
the preliminary baseline budget for the seismic upgrade but through
diligent teamwork, options review and an Independent Review, a revised
baseline was approved with NA-11 characterizing the process as a "model
for others to follow". SCIF identified cost issues that resulted in an
aggressive corrective action plan that reallocated costs to appropriate
cost centers and to delete some scope without compromising the projects
integrity. The project is currently on track to be completed within
budget and on schedule.

The National Ignition Facility (NIF) project continued in accordance
with plan and all FY 2003 milestones were completed on or ahead of
schedule. "First Light" and "First Light to Target Chamber" were
accomplished as well as completion of laser beampath enclosures,
installation of first target area diagnostics and completion of
activation and commissioning of the four NIF Early Light laser beams.
LLNL also did well in developing and implementing plans for optimal use
of NIF. The NIF Director is working to build a national program that
will support the requirements of NIF's stakeholders and optimally
utilize the resources of the Laboratories. However, continuing effort is
needed to fully integrate a national program that addresses the needs of
NNSA.

LLNL managed their hydrotest program well. However, further attention by
UC is needed to fully develop an integrated plan between the two
laboratories with particular emphasis on the conduct of LANL experiments
at LLNL facilities.

**[OPERATIONS]{.underline}**

LLNL exceeded their facility short and long range goals in support of
mission requirements. The aggregate availability of their Readiness in
Technical Base and Facilities (RTBF) for FY 2003 was 99.5% with no
negative impacts on meeting programmatic milestones. The Laboratory also
met or exceeded the goals in the FY 2003 Ten Year Comprehensive Site
Plan (TYCSP) and has already met the FY 2005 goal of stabilizing the
deferred maintenance backlog with their maintenance management system
and prediction tool being used as the NNSA model.

Progress continued in implementing Integrated Safety Management (ISM) as
evidenced by the Work Smart Standards for Safety Basis Requirements for
Nonnuclear Facilities at LLNL. Training is being developed and an
implementation plan completed. NNSA did issue a Preliminary Notice of
Violation to LLNL this fiscal year due to deficiencies in operational
radiological controls and a failure to follow established work controls
that led to an overexposure incident in FY 2002. On an institutional
basis, LLNL demonstrated some improvements in the identification of
hazards function, an area of weakness identified by the LSO and the
SEMI, through the implementation of the electronic Integrated Work Sheet
(e-IWS). However, weakness in the feedback for improvement function,
also identified by the LSO and the SEMI, still requires attention from
Laboratory management to ensure complete and effective corrective
actions.

LSO is concerned with the increased number of safety related occurrences
in FY2003. Although no serious injuries have resulted from these
incidents, this trend is of concern and, should it continue, the
potential for serious injury increases. Increased management attention
is needed to address these continuing worker safety related events and
to aggressively identify corrective actions that will reduce/eliminate
them.

In the area of Nuclear Facility Safety, the Laboratory performed
satisfactorily. While submittals to meet the April 10, 2003 compliance
deadline (10CFR830, Subpart B) for Documented Safety Analyses (DSAs)
were met, progress was lacking in implementing the Unreviewed Safety
Question (USQ) procedure. The Laboratory's safety basis amendments had a
large variation in quality.

Safeguards and Security has begun implementing Integrated Safeguards and
Security Management (ISSM) and met all of the established milestones for
FY 2003. There are three findings with regards to LLNL's implementation
of ISSM dealing with lines of responsibility for Communications Security
(COMSEC), a lack of systematically integrating safeguards and security
management and work practices to prevent recurring security incidents,
and insufficient measures and controls for managing risks related to
security incidents. LLNL's cyber security and Material Control and
Accountability (MC&A) programs remained strong and the annual LSO survey
resulted in a Satisfactory rating, the highest possible. LLNL's
performance tests were also rated satisfactory by both LSO and by the
DOE Office of Independent Oversight and Performance Assurance (OA). LLNL
did an excellent job in securing the closure of East Avenue, has
responded well to the numerous SECON conditions that place the
laboratory at heightened security, and has done well in addressing the
requirements of the current Design Basis Threat (DBT).

In the area of physical security, a series of security incidents
involving the LLNL Protective Force demonstrate that deficiencies exist
in training, communication, roles and responsibilities, unit
cohesiveness as well as implementation of continuous improvement under
ISSM. Although at no time was there any direct threat to sensitive
material or information, the nature of these incidents leaves open the
potential for such risk. Further management attention is required to
resolve and prevent further recurrence of these issues. In addition,
external reviews identified discrepancies in incident reporting,
security key control and inventory, and accountability of classified
removable electronic media indicating the need for LLNL to strengthen
its self-assessment efforts. LSO rates LLNL as satisfactory in this
area. However, this is a low satisfactory and significant management
attention is needed to ensure improvement.

LLNL has accomplished all they can with regards to working with NNSA to
develop a long term plan to reduce inventories of surplus and excess
Special Nuclear Material (SNM) and waste. However, the plan has yet to
be implemented by NNSA and until such time it is not possible to fully
evaluate the effectiveness of the lab's work. LSO considers the effort
to be excellent with the overall outcome pending.

The environmental program at LLNL exceeded all their goals for waste
treatment and disposal as well as meeting regulatory requirements.  In
addition, LLNL completed its DWTF milestones ahead of schedule.  The
environmental restoration project completed all enforceable milestones
ahead of schedule and within funding.

The Laboratory's emergency management program continued to show
improvement. LLNL had an approved Project Management Plan, submitted
timely required updates to the Emergency Plan that were considered
acceptable, and submitted six Emergency Preparedness Hazards Assessments
for LSO approval. An exercise conducted in May 2003 demonstrated
adequate capability for managing a potential emergency. Several
weaknesses and one deficiency were identified and a corrective action
plan was submitted to LSO.

Business systems at the Laboratory are considered outstanding,
exhibiting cost effective business and financial systems and controls.
Appropriate management control systems are in place and functioning as
intended. Cost accounting practices were sound, effective and support
federal stewardship objectives. An Ernst and Young assessment as well as
an internal audit self-assessment identified enhancements to the
existing business practices and systems, most of which have been
implemented. LLNL reviewed the lessons learned from the various LANL
audits and proceeded to validate that their internal control systems are
in place. LLNL has consistently been responsive to new and changing
DOE/NNSA financial, procurement and property management requirements as
well as benchmarking internally with other DOE/NNSA facilities and with
commercial entities, to improve business practices.

LLNL continued to provide the skills necessary to enhance the science
base. This was accomplished through implementation of the annual
workforce review to identify skill gaps and workforce issues, completion
of initiatives responsive to employee survey findings and an
institutional effort to incorporate the UC image in addressing pipeline
recruitment needs. In addition, LLNL utilized their self-assessment
process to provide the status of its efforts in ensuring core and
critical skills requirements are identified and monitored to provide
continuity from Appendix O. There continues to be some concern over the
absence of a system at LLNL to track declined job offers, which impacts
its reporting on the semi-annual metrics for critical skill candidates.
The Livermore Site Office and Laboratory are continuing to explore how
LLNL can be responsive to these reporting requirements.

The Laboratory has developed a comprehensive mid-level leadership and
management development strategy that provides for the developmental
needs of supervisors and managers as well as potential future leaders.
LLNL's Leadership and Management Development Program offered multiple
general management courses in-house, sponsored leadership/career
development programs specifically addressing issues unique to high
potential minorities, maintained a cadre of external courses for various
levels of leader and offered a highly popular Leadership Lecture Series
open to all employees. In addition, over 300 employees identified as
having high potential have participated in Directorate Leadership
programs with over 45% having been promoted or having assumed increased
levels of responsibility since completing the programs.

At the more senior management level, the Management Institute is a very
successful program for "preparing the next generation of Laboratory
Leaders". The Institute is a two and one-half day program provided
annually, with the goal of broadening the participants' "understanding
of current issues facing the Lab, institutional management
responsibilities, and strategies for leading the Laboratory into the
future." Subsequent to the Institute, the Director's Office continues to
monitor the development of alumni by providing guest speakers and
seminars, and through utilizing them as a cadre for managing special
projects with broad institutional implications.

LLNL's efforts to sustain community initiatives continue to be at the
outstanding level. Community relations is a high priority with LLNL
which has an extensive outreach program through their Science and
Technology program as well as the Public Affairs Office that addresses
California science and education goals and key community initiatives.
LLNL has also entered into mutual-aid agreements with surrounding
communities for the operation of an on-site fire dispatch center and
provides professional fire department emergency response capability to
the surrounding community through the State of California mutual aid
program.

  -------------------------------------------------------- ---------------
  Overall LLNL Rating                                      **Excellent**

  -------------------------------------------------------- ---------------

  ------------- --------------------------------------------------- -----------------
  **Mission**                                                       **Outstanding**

  **1.**        Develop and Implement a Common UC Design Laboratory **Excellent**
                Certification Strategy                              

  **2.**        Develop with NNSA and implement long-term balanced, **Outstanding**
                integrated stewardship                              

  **3.**        Develop with NNSA and implement near-term balanced  **Outstanding**
                weapon program                                      

  **4.**        Develop and implement sound non-proliferation /     **Outstanding**
                counter terrorism program basis                     

  **5.**        Enhance and nurture a strong science and technology **Outstanding**
                base in support of NNSA national security           
                objectives                                          

  **6.**        Achieve successful completion of projects and       **Outstanding**
                development of user facilities                      
  ------------- --------------------------------------------------- -----------------

  ---------------- --------------------------------------------------- ------------------
  **Operations**                                                       **Satisfactory**

  **7.**           Maintain a secure, safe, environmentally sound,     **Satisfactory**
                   effective and efficient operations and              
                   infrastructure basis in support of mission          
                   objectives                                          

  **8.**           Utilize UC strengths to recruit, retain and develop **Excellent**
                   the workforce basis                                 

  **9.**           Sustain effective Community Initiatives             **Outstanding**
  ---------------- --------------------------------------------------- ------------------

**Detailed Appraisal Results**

**Mission**

  ---------------------------------------------------------- ---------------
  **Performance Objective 1**                                **Excellent**

  **Develop and Implement a Common UC Design Laboratory      
  Certification Strategy**                                   
  ---------------------------------------------------------- ---------------

  ---------------------------------------------------------- -----------------
  **Performance Measure 1.1**                                **Outstanding**

  **Develop an integrated, scientifically-based quantitative 
  certification methodology that has been externally         
  reviewed for use in future warhead certification and to    
  support the Annual Certification Process**                 
  ---------------------------------------------------------- -----------------

LANL and LLNL have demonstrated an outstanding capability to work
together in the QMU area to develop a common framework which is being
developed into a formal certification methodology.  As the laboratories
work out their own examples and applications to their problems, they are
finding large areas of similarity in approach.  The development of a QMU
framework has also provided a useful mechanism for discussing priorities
for the balance of the stockpile stewardship program, in particular such
issues as priorities for plutonium and high energy density physics
experiments.  Common nomenclature and approach for QMU was articulated
during the Risk Panel preparation for Confidence Conference in Omaha and
that advance in the methodology facilitated reaching common positions on
issues addressed by the Risk Panel.  The methodology has been generally
accepted by outside organizations, including the STRATCOM advisory
panel.

Los Alamos and Lawrence Livermore are to be commended not only for the
result of establishing QMU methodology, but for the excellent
collaboration between the two competing laboratories in this critical
area that is so central to stockpile stewardship..  They are encouraged
to continue this cooperation.

  ---------------------------------------------------------- ---------------
  **Performance Measure 1.2**                                **Excellent**

  **Demonstrate application of a common assessment           
  methodology using Quantification of Margins and            
  Uncertainty (QMU), in major warhead assessments**          
  ---------------------------------------------------------- ---------------

LLNL performance supports a High Excellent. They have shown initiative
in applying QMU to warhead assessments and SFIs this past Fiscal Year
including the W87 Alt 342, production liaison, the W80 LEP and in peer
review of several issues raised on LANL systems. QMU has been
successfully used by LLNL leading to a recommendation related to
component reuse. **\
**

  ---------------------------------------------------------- ---------------
  **Performance Measure 1.3**                                **Excellent**

  **Demonstrate progress toward quantifying margins and      
  reducing uncertainties relevant to primary and secondary   
  performance**                                              
  ---------------------------------------------------------- ---------------

The Secondary Certification campaign was originally laid out by LLNL
with a QMU-like approach in mind in the development of a secondary
certification capability and the goals of the primary campaign are also
consistent with the goals of QMU. The pace of required experiments has
been slower than desired because of resource constraints in the primary
and secondary certification campaigns. The recently completed Piano
experiment has returned excellent data that will provide new information
relevant to developing models for primaries. Validation experiments
conducted at LANSCE area C have also provided data to increase the
confidence in models used to make decisions relevant to critical
life-extension programs.

  ---------------------------------------------------------- -----------------
  **Performance Objective 2**                                **Outstanding**

  **Develop with NNSA and implement long-term balanced,      
  integrated stewardship**                                   
  ---------------------------------------------------------- -----------------

  ---------------------------------------------------------- -----------------
  **Performance Measure 2.1**                                **Outstanding**

  **Support the needs of warhead assessment and              
  certification by coordinated programs of targeted small-   
  and large-scale experiments and mining of archival UGT     
  data to improve predictive capability**                    
  ---------------------------------------------------------- -----------------

LLNL performance has been Outstanding. LLNL has had to overcome
unanticipated obstacles to conduct these small and large scale
experiments at Contained Firing Facility and the Nevada Test Site.
Almost all planned experiments were successfully conducted. These
include among others, large scale and small scale hydros. LLNL is also
working on a cooperative basis with LANL to use DARHT and vice versa to
make LLNL CFF available to support LANL.

  ---------------------------------------------------------- -----------------
  **Performance Measure 2.2**                                **Outstanding**

  **Demonstrate advances in radiography technology and       
  develop joint options and recommendations for future x-ray 
  and proton radiographic capability that support the        
  quantitative certification methodology**                   
  ---------------------------------------------------------- -----------------

LLNL has made great progress in improving radiographic capabilities. The
most notable success has been the meticulous planning associated with
Beryllium control and clean-up at the contained firing facility.
Demonstration that beryllium containing hydrotests can be safely
executed at CFF is a major accomplishment. Recent developments in
methods for validation of primary design codes with test problems that
have analytic solutions have contributed to understanding the
computational resolution necessary to achieve stated levels of accuracy
for ASCI design codes. This provides independent validation of models
that are being used to determine the experimental resolution that would
be required for an advanced radiographic facility.

  ---------------------------------------------------------- ---------------
  **Performance Measure 2.3**                                **Excellent**

  **Demonstrate ASC simulation and modeling capabilities     
  that support the ongoing needs of stockpile assessment and 
  certification**                                            
  ---------------------------------------------------------- ---------------

Concur with self assessment. However, we would like to emphasize several
additional points. LLNL\'s contributions to the JASON study of the ASC
program were essential to the constructive and positive outcome of the
study. Their responsiveness to questions from both the headquarters and
JASONs were timely, comprehensive and deserving of recognition.
Computing on the White machine continues to be the platform of choice
for the most challenging codes across the tri-lab. On a less positive
note, LLNL has yet to develop a credible Validation program, which is
central to developing predictive code capabilities. However, there has
been impressive progress in Verification, in particular new and unique
solvable models with can be used to verify codes at LLNL and LANL. The
program would be significantly strengthened if V&V efforts were more
closely integrated with the overall QMU strategy at LLNL.

  ---------------------------------------------------------- -----------------
  **Performance Measure 2.4**                                **Outstanding**

  **Improve and apply tools and models for prediction of     
  systems and/or component lifetimes**                       
  ---------------------------------------------------------- -----------------

NNSA concur with the Laboratory's self assessment. Additionally, LLNL is
making good progress on science issues of interest to basic weapons
performance. Recent accomplishments include energy balance and
predictive high explosives models. Modeling efforts are well connected
to the code projects, which enable rapid improvement in code
capabilities.

  ---------------------------------------------------------- -----------------
  **Performance Measure 2.5**                                **Outstanding**

  **Develop and implement a collaborative and complementary  
  program of experiments at High Energy Density (HED)        
  facilities that supports the quantitative certification    
  methodology**                                              
  ---------------------------------------------------------- -----------------

LLNL technical performance in the areas of ignition and HED weapons
physics has been outstanding in the past year. There have been numerous
technical successes in these areas. Several particular examples deserve
notice. First, recent LLNL work that has expanded the phase space
available for indirect drive ignition has been outstanding and of major
importance. Our confidence in indirect drive ignition has increased
significantly as a result of this work. Secondly, LLNL executed the
first hohlraum plasma characterization experiment on NIF. This
experiment was a landmark event and effectively ushered in the era of
"NIF use" for the stewardship program. The technical coordination
between the ICF Program and the NIF Project required to execute this
experiment was formidable. The successful execution of this experiment
testifies to LLNL's outstanding technical and management capabilities.
Numerous high quality and high visibility experiments in the HED area
were also executed in the past year. All major milestones planned for FY
2003 were met.

  ---------------------------------------------------------- ---------------
  **Performance Measure 2.6**                                **Excellent**

  **Develop an integrated program for plutonium capabilities 
  of LANL and LLNL to support the overall NNSA strategic     
  requirements**                                             
  ---------------------------------------------------------- ---------------

The reported efforts only go part way to meeting the intention of the
performance objective and overlook the issues involved in developing a
coordinated and prioritized plan for experiments using plutonium. The
broad issue that requires senior level laboratory management and
coordination between the two University of California laboratories is
the development of a long-term plan to balance costs and priorities for
the broad suite of plutonium facilities, capabilities and activities.
This includes facilities maintenance, including safety, safeguards and
security; plutonium science requirements to support the development of
predictive nuclear performance models, pit manufacturing, surveillance
tests, engineering tests; and integrated tests including Nevada Test
Site activities.

  ---------------------------------------------------------- -----------------
  **Performance Objective 3**                                **Outstanding**

  **Develop with NNSA and implement near-term balanced       
  weapon program**                                           
  ---------------------------------------------------------- -----------------

  ---------------------------------------------------------- -----------------
  **Performance Measure 3.1**                                **Outstanding**

  **As part of the Annual Certification Process, the         
  laboratory directors will complete the annual assessments  
  of the continued safety, reliability and performance of    
  all warhead types in the stockpile including whether       
  nuclear testing is required for resolution of any issue;   
  and support DOE as required during interagency and         
  community coordination of the Annual Certification         
  Process**                                                  
  ---------------------------------------------------------- -----------------

LLNL successfully met all annual assessment milestones. They also added
additional members for the RED Team from LANL and SNL for independent
review and validation of the annual assessment. Assessments were
successfully reviewed with STRATCOM Scientific Advisory Group/stockpile
Assessment Team and with the NNSA.

  ---------------------------------------------------------- ---------------
  **Performance Measure 3.2**                                **Excellent**

  **Provide technical support to production complex          
  operations, including the Integrated Weapons Activity Plan 
  (IWAP)**                                                   
  ---------------------------------------------------------- ---------------

LLNL provided excellent support to Pantex and met most objectives and
milestones. LLNL provided timely review of weapon response documents
included on the IWAP without any major LLNL performance issues. LLNL
provided expert technical support as Nuclear Explosive Safety Study
members and in many other safety studies to support continuing
activities at Pantex.

  ---------------------------------------------------------- -----------------
  **Performance Measure 3.3**                                **Outstanding**

  **Deliver on W88 Pit Manufacturing and Certification       
  Project major milestones**                                 
  ---------------------------------------------------------- -----------------

LLNL\'s primary function with respect to the pit project is in the
support of plutonium operations that are not currently achievable at
LANL (environmental testing) and to provide a peer process capability to
validate the overall certification methodology as well as validation of
specific simulations and analyses which are vital to the certification
process. LLNL has provided outstanding support in both areas despite the
fluctuations in the availability of budget to perform the scope of work
that was originally negotiated. In addition, LLNL has provided
consistent representation at monthly project status meetings and has
been responsive in dealing with issues broached at those meetings. A few
of the strengths that LLNL has exhibited in the support of the pit
project are discussed below:

Excellent and timely technical support of engineering experiments - LLNL
completed the compatibility experiment on the DEV 2 pit in the fourth
quarter FY 2003. LLNL has also provided regular and adequate
interactions in the planning for the shock and vibration test to be
performed at LLNL Superblock Bldg. 334.

LLNL developed and executed reasonable and adequate plans to conduct
peer review activities. In FY 2003, LLNL developed a comprehensive plan
to review and assess the appropriate vital aspects of LANL\'s W88 pit
certification methodology. Due to funding constraints in the pit
certification budget, LLNL was forced to replan its activities and
provided a useful review, including an excellent confirmatory analysis
of pit performance fluctuations caused by variations in manufacturing.
LLNL also provided an excellent review of the revised engineering
certification plan.

Developed work packages to mirror LANL project controls system - LLNL
developed a set of work packages for planned physics and engineering
work that will seamlessly flow into the project controls system
developed for the pit project.

LLNL was instrumental in resolving issues at the Device Assembly
Facility at the NTS and at Superblock.

LLNL has provided excellent support to the Pit Campaign in technology
development for the Modern Pit Facility and in pit manufacturing
capability. For example, LLNL provided critical support in a study
evaluating the best method for plutonium purification in the Modern Pit
Facility.

  ---------------------------------------------------------- -----------------
  **Performance Measure 3.4**                                **Outstanding**

  **Deliver on the major milestones for the Life Extension   
  Programs for the W76, the B61-7/11, and the W80- 2/3 in    
  accordance with the Phase 6.X process**                    
  ---------------------------------------------------------- -----------------

LLNL has made significant progress in accomplishing the objectives of
the W80-3 LEP by overcoming residual fallout from the FY 2002
Congressional holdback and resolving issues associated with Air Force
(AF) funding and support. The hydrotest program required to establish a
performance baseline and to address LEP design changes has been
successful to date and LLNL personnel have been responsive and
supportive. NNSA anticipates receiving more details supporting the cost
re-baseline necessitated by the Congressional hold and negotiation of a
revised FPU with the AF.

  ---------------------------------------------------------- -----------------
  **Performance Measure 3.5**                                **Outstanding**

  **Conduct stockpile surveillance and assessment            
  activities, including investigation and subsequent         
  resolution of significant findings on a priority basis,    
  and issues identified in technical assessment reports**    
  ---------------------------------------------------------- -----------------

In FY 2003, LLNL made significant progress in closing outstanding SFIs.
LLNL has successfully met its design and production agency core
surveillance obligations and continues to integrate the surveillance
database with the overall Directorate data management system.

  ---------------------------------------------------------- ---------------
  **Performance Measure 3.6**                                **Excellent**

  **Support directive schedule requirements**                
  ---------------------------------------------------------- ---------------

Livermore continues to provide excellent support for the W87 Life
Extension Program. Specific activities include on-site support at Y-12
and continued support of Mechanical Safe and Arm Device production at
KCP

  ---------------------------------------------------------- -----------------
  **Performance Objective 4**                                **Outstanding**

  **Develop and implement sound non-proliferation / counter  
  terrorism program basis**                                  
  ---------------------------------------------------------- -----------------

  ---------------------------------------------------------- ---------------
  **Performance Measure 4.1**                                **Excellent**

  **Sustain and expand intelligence and counter-intelligence 
  programs analysis and analytical data systems for          
  detecting and thwarting Proliferation and Terrorism**      
  ---------------------------------------------------------- ---------------

The Counterintelligence Office (CI) Security Awareness for Employees
(SAFE), Lawrence Livermore National Laboratory (LLNL conducted an
Excellent CI program for FY 2003.

The SAFE Analysis Program conducted eight in depth major analytical
projects in addition to supporting counterintelligence officers (CIO)
with assessments of graphic depiction in support of SAFE
CI/Counterterrorist (CT) investigations. Several of the analytical
projects assisted in establishing SAFE's 2003 priorities, led to
counterintelligence investigations, and provided Laboratory management
with information upon which to base specific decisions related to the
management of risk.

SAFE's CT effort is highly pro-active and integrated into the four
substantive CI programs Communications and Awareness, Investigations,
Information Special Technologies and Analysis. SAFE has prepared
Terrorism Threat Briefings and provided them to a wide range of LLNL
audiences from classified senior manager sessions to unclassified summer
intern gatherings. During the third quarter, SAFE CIOs completed local
threat briefings of LLNL senior management of all 13 Lab Directorates.
Titles of these specially developed briefings were: *How to Catch
Terrorists, In Pursuit of Justice, Inside the Terrorist Mind, Terrorist
Threats to the United States, Terrorist Funding, UNABOM, University and
Airline Bombings, and the Search for Eric Robert Rudolph.* SAFE
personnel also upon special request provided Terrorism Threat Briefings
to NNSA audiences at Los Alamos National Laboratory, Sandia National
Laboratory and Pacific Northwest National Laboratory. During 2003, SAFE
continued providing CT briefings on a routine basis to the roll-call
meetings of the Protective Force Division (PFD). SAFE has introduced the
local site FBI agent to the PFD briefing lineup and has received highly
commendable comments from PFD officers as a result of this initiative.
In the near future, SAFE and the FBI will initiate unclassified "CT
handbook" for each PFD officer, which will further their training,
enhance their understanding, and further cement the intelligence/law
enforcement relationship derived from this liaison. SAFE's
Communications and Awareness Coordinator provided a briefing to the LLNL
Export Control Panel, reaching eight recipients. Additionally, the
Coordinator distributed three issues of Tech Transfer Notes, ten Air
Force Office of Special Investigations' Terrorism Bulletins and one
terrorism threat matrix.

SAFE's CIO/Technical Expert (TE) continued on with a highly pro-active
and effective cyber program. The CIO/TE was instrumental in reviewing
close to 126 computer security plans for Sensitive Country Foreign
Nationals in addition to working a number of substantial investigations
with a cyber component necessitating the acquisition and review of hard
drives, computer disks, and emails. Additionally, the CIO/TE
participated in briefings of systems administrators and Directorate
Management with the Senior Counterintelligence Officer (SCIO). SAFE and
the Lab Computer Security Office (CSO) have worked closely to ensure
that Laboratory managers are aware of the risk involved in the cyber
access for foreign nationals and sensitive country foreign nationals
(SCFN). Both SAFE and the CSO have completed draft reports regarding
cyber security and CI risks associated with cyber access by foreign
nationals.

SAFE is recognized as the focal point for the collection of intelligence
and law enforcement information that might be relevant to terrorist
interest at LLNL. It's liaison continued to be extensive and exemplary.
In addition to a close working relationship with officials from the San
Francisco Field Office of the Federal Bureau of Investigation (FBI),
SAFE hosted high ranking officials from the FBI Headquarters,
Washington, DC. SAFE was the first DOE/NNSA Lab to co-locate an FBI
agent within the CI program. The Safe relationship with the FBI
continues to be outstanding.

SAFE supports the Counterintelligence Evaluations Program (CIEP) through
conducting excellent personnel security file reviews in excess of 123
for FY 2003. The personnel file reviews enabled CIEP to complete its
counterintelligence evaluations.

SAFE produced 56 Intelligence Information Reports (IIR), significantly
more than any other CI Office within the Office of Defense Nuclear
Counterintelligence (ODNCI). The IIRs were in response to requirements
that have been developed by the Office of Counterintelligence (OCI) and
by the members of the USIC. The IIRs or their contents were
provided/distributed to NNSA Senior Management and the United States
Intelligence Community (USIC) and were well received.

SAFE has met or exceeded all of the Investigations Programs Performance
Objectives. All Investigations complied with pertinent laws, orders,
regulations as well as the Investigations Procedural Guide. All
investigations were substantively sufficient and conducted in a timely
manner.

SAFE met or exceeded all of the CI Performance Objectives for the field
CI Offices that were developed by the ODNCI/OCI.

For the intelligence portion of this measure, the performance of NAI's
International Assessments Program (Z-Division) during this period was
Outstanding. Their exceptional wide spectrum and high quality
intelligence activities have been solidly established to anticipate
threats, react quickly to existing situations, and execute a very
proactive program in support of the overall intelligence community. In
addition to maintaining its traditional role in foreign nuclear weapons
programs and weapons designs assessments, Z-Division has rapidly
expanded its core competencies in developing a comprehensive chemical
and biological weapons assessment program, and a significant capability
in information operations. During these very challenging times, they
continue to make excellent headway in understanding and dealing with
nuclear proliferation issues related to rogue states, terrorist groups,
and sub/transnational entities.

Z-division's unique work continues to standout as exceptional in
comparison to activities conducted at other sister laboratories (LANL,
SNL, PNNL). Their analysts apply the broadest range of
multi-disciplinary expertise, and stay consistently focused while
maintaining a holistic view of the topic and issue, all to the
satisfaction of the sponsors. As such, many very creative innovations
continue to be developed to significantly enhance the capabilities of
the intelligence community.

LLNL continues to perform at a very high level in support of the NNSA
nonproliferation export control mission. Major accomplishments this year
include, e.g.: review of all U.S. WMD-related exports for proliferation
concern, principally in the areas of nuclear technology and bona fides
of end-users; support for U.S. inspections in Iraq, such as development
of a Iraq-related database for the IAEA and on-the-ground support for
IAEA and Iraq Survey Group activities; technical coordination of all
international export control assistance programs in Russia; completion
of several Proliferation Risk and Analysis Project studies, designed to
anticipate export control vulnerabilities.

Issues and Concerns:

ODNCI has no operational concerns with SAFE. SAFE is in the process of
identifying and then will be hiring replacements for a full time CIO and
a Principal Information Systems Assistant due to the retirement and a
long-term medical issue of former staff members.

For Z-division, there continues to be a struggle to recruit sufficient
staff to meet the growing demands imposed by DOE/IN and the various IC
sponsors.

The only outstanding issue with LLNL concerns the timely completion of
tasked technical and proliferation risk studies. LLNL has been apprised
of our concern and will take appropriate steps to address it.

  ---------------------------------------------------------- -----------------
  **Performance Measure 4.2**                                **Outstanding**

  **Sustain and expand international cooperative programs to 
  reduce the threat of nuclear proliferation**               
  ---------------------------------------------------------- -----------------

Primary support contractor for first-of-a-kind transparency
implementation program in closed nuclear cities in Russia, since 1994,
in fulfillment of the mission to implement transparency activities
associated with the purchase of Russian uranium by the U.S. Develops
procedures for monitoring in Russia, including development of
non-destructive assay equipment to measure U235 in various containers in
process areas of Russian facilities. Also supports data management,
monitor training, health and safety, and quality assurance.

In 2001-2002, in cooperation with Oak Ridge National Laboratory,
developed and in 2003 now maintains data base exceeding 60,000 pages of
input from Russian data, with capability for archiving and retrieving
information at eleven facilities. In 2003 provided technical evaluation
of data on a classified level that provides important information for
establishing assurance that the program's objectives are being met.

LLNL provided continuous dosimetry and bioassay support for personnel
traveling from the U.S., including staff from other laboratories and
contractors. Maintains data base of sensitive medical and personal
information and prepares quarterly and annual updates on health physics.

In 2002 developed updated non-destructive assay (NDA) measurement
equipment and in 2003 shipped the equipment to the first of the four
sites to replace outdated units

The Laboratory has done an outstanding job in the development,
installation and technical support of the two Blend Down Monitoring
Systems (BDMS) currently installed in Russia. In support of this effort,
the laboratory has provided technical expertise in the area of health
physics that resulted in a \$125,000 cost saving to the program and
helped the Program gain a better understanding of Russian radiation
exposure requirements.

The Laboratory has an active Continual Improvement process for
assessing, developing and implementing technologies to better monitor
weapons-grade HEU in Russia. The information obtained from this
instrumentation is a key technical input towards assessing the
non-proliferation goals of the HEU Transparency Program. The new, easier
to-use Non-destructive Assay instrumentation has been fielded in two of
the four Russian process facilities involved in the HEU Transparency
Program.

  ------------------------------------- ---------------- -----------------
                                                         

  FCFMS                                 \$82,500         Excellent

  NCI nonproliferation program in       \$1,750,698      Excellent
  Russia for scientist engagement and                    
  downsizing the Russian nuclear                         
  weapons complex.                                       

  INECP                                                  Excellent

  IAEA Safeguards, Physical Protection  \$900,000        Outstanding
  and Nonproliferation Policy Support                    

  DPRK Safeguards                       \$200,000        Outstanding

  Sustainability of Safeguards and      \$140,000        Outstanding
  Security Systems in the NIS/Baltics                    

  Iraq Inspections                      \$600,000        Outstanding

  Nuclear Noncompliance Verification    \$1,300,000      Outstanding
  ------------------------------------- ---------------- -----------------

[Office of Fissile Materials Disposition (NA-26) Performance Evaluation
of LLNL for FY 2003]{.underline}

[Packaging, Storage and transportation]{.underline} - Work has just
begun in this area, but progress is being made under very difficult
changing policy swings.

. **Quality of Technical Support** - Outstanding

**Relevance to the Mission** - Excellent

**Management Effectiveness** - Outstanding

[Waste Management]{.underline} - This area has changed, but is follow-on
to what was done by LLNL in the past. Again, progress is being made
under very difficult policy changes.

**Quality of Technical Support** - Outstanding

**Relevance to the Mission** - Excellent

**Management Effectiveness** - Outstanding

[Monitoring and Inspection]{.underline} - LLNL supports the Joint
Monitoring and Inspections Working Group (JMIWG), established by the
Joint Consultative Commission (JCC) for the Agreement Between the
Government of the United States of America and the Government of the
Russian Federation Concerning the Management and Disposition of
Plutonium Designated as No Longer Required for Defense Purposes and
Related Cooperation (PMDA), is developing an agreed set of detailed
measures, procedures, and administrative arrangements for M&I Activities
under the PMDA.

LLNL's lab representative functions as a member of a technical advisory
body that serves as the principal mechanism to DOE for the
identification, analysis and evaluation of technical aspects,
procedures, and costs for M&I measures associated with the disposition
of weapons-grade plutonium. As required, they also manage contract work
with Russian counterparts for work done by Russian institute in support
of M&I activities. LLNL overall performance is Outstanding.

Issues and Concerns:

Shipment of the new NDA units was not timely, and delays of more than
three months occurred. The problem was resolved by year's end.

No concerns or issues for the HEU Transparency Program.

  ---------------------------------------------------------- -----------------
  **Performance Measure 4.3**                                **Outstanding**

  **Develop and expand complex systems modeling to enhance   
  prediction and identification of threats, prioritization   
  and integration of counter-terrorism efforts, and          
  effectiveness of response systems for terror events**      
  ---------------------------------------------------------- -----------------

NAI's Proliferation Detection and Defense Systems Program (Q Division) ,
specifically its Counterproliferation Analysis and Tactical Systems
Sections, have performed in an Outstanding fashion for the areas related
to this measure. The most noteworthy area is that related to the well
established CAPS program and the more recent DHS supporting activities
applicable to various targets of interest. CAPS has been very
successfully expanded to include the Signatures Program and HOPS, both
of significant utility to our national defense and security forces. In
addition to success of CAPS, JCATS and its spin-off ACATS, is also
proving to be extremely valuable to the U.S Joint Forces. Based upon
sponsor acceptance of these systems, it is clear that this work by Q
Division has reached a preeminent position in this arena.

NARAC support to harmonization including exercise modeling support has
been Outstanding. NARAC products far exceed the products of
international partners and are recognized as "best in class" in
international fora.

NOTE: While activities being performed for the International Emergency
Management and Cooperation (IEMC) program are not specifically
identified in the PEP, IEMC utilizes NARAC resources to support
harmonization of international atmospheric modeling.

Issues and Concerns:

Balance between Q Division's operational military support commitments
and its conduct of R&D continues to be an issue that the Division Leader
is successfully tackling.

  ---------------------------------------------------------- -----------------
  **Performance Measure 4.4**                                **Outstanding**

  **Develop and transition technologies for large-scale      
  deployment for civilian preparedness against terrorist     
  biological, chemical and other attacks**                   
  ---------------------------------------------------------- -----------------

The collaborative effort with LANL in deploying BASIS, and then DHS
BioWatch unto itself exemplifies outstanding performance. The related
work involving NARAC/LINC further complements this effort. APDS
deployment has also been a great success in multi-agent bio detection.
In the chemical countermeasures arena, the FSC and Microtechnology
Center excel in their performance. Attaining OPCW Final Accreditation to
support the CWC is an Outstanding accomplishment.

Issues and Concerns:

Same issue as 4.3, where the balance between operational support to the
user community and conduct of R&D is in jeopardy.

  ---------------------------------------------------------- -----------------
  **Performance Measure 4.5**                                **Outstanding**

  **Develop and demonstrate nuclear detection and monitoring 
  technologies; provide technologies and expertise to        
  enhance protection of nuclear materials in, at, and        
  outside of US borders; and maintain the capability to      
  deploy a nuclear emergency response team for protecting US 
  assets from radiological and nuclear threats**             
  ---------------------------------------------------------- -----------------

The RDC work for radiation detection related to smuggling has been
outstanding with contributions by the PAT Directorate especially
noteworthy. Additional activities related to nuclear threat assessment
conducted by NAP, nuclear incident response involving RAP/ARG/NEST/JTOT,
nuclear detection systems including DTS/UNWD/ICONEX, and nuclear
attribution all have been exceptional.

The performance of LLNL has been outstanding in support of the NNSA
Office of Nonproliferation Research and Engineering during the
assessment period under review. LLNL's technical support and
deliverables rank consistently high, specifically in the areas of
hyperspectral imaging, synthetic aperture radar, radiation detection
R&D, and remote sensing and seismic monitoring.

Issues and Concerns:

The balance of operational commitments versus conducting essential R&D
is also an issue for this measure.

  ---------------------------------------------------------- -----------------
  **Performance Measure 4.6**                                **Outstanding**

  **Develop global situational awareness with the defense    
  and intelligence communities and enable the necessary      
  technical underpinnings to monitor and track facilities,   
  people, and situations worldwide in real-time**            
  ---------------------------------------------------------- -----------------

NAI's Proliferation Detection and Defense Systems Program (Q Division),
specifically its Proliferation Detection Section, have performed in an
outstanding fashion for the areas related to this measure. The
activities related to the LASI Falcon Talon, and the SATRN especially
stand out as being exceptional work. The innovations of the Eyeglass
project have made significant contributions to national planning for
space exploration activities. The seismic monitoring collaborative work
with other national labs, and activities involving information
operations (IOAC/LILAC) have been outstanding.

The performance of LLNL has been outstanding in support of the NNSA
Office of Nonproliferation Research and Engineering during the
assessment period under review. LLNL technical support and deliverables
rank consistently at a high level, specifically in the areas of
hyperspectral imaging, synthetic aperture radar, radiation detection
R&D, and remote sensing and seismic monitoring.

  ---------------------------------------------------------- -----------------
  **Performance Objective 5**                                **Outstanding**

  **Enhance and nurture a strong science and technology base 
  in support of NNSA national security objectives**          
  ---------------------------------------------------------- -----------------

  ---------------------------------------------------------- -----------------
  **Performance Measure 5.1**                                **Outstanding**

  **Develop and implement an integrated and balanced         
  strategy for investing LDRD, programmatic and              
  institutional resources to ensure the long-term vitality   
  of the Laboratory Science and Technology base to support   
  the NNSA mission and emerging national needs**             
  ---------------------------------------------------------- -----------------

LLNL has performed at an ***outstanding*** level in developing and
integrating a balanced strategy for investing programmatic and
institutional resources to ensure the long-term vitality of LLNL science
and technology base while supporting the DOE/NNSA mission, DHS mission
and emerging national needs.

LLNL institutional resources include the Laboratory Directed Research
and Development (LDRD) Program, Work For Others (WFO) Program,
Technology Transfer, programmatic and institutional investments to
ensure the long-term vitality of the Laboratory's science and technology
base.

During FY 2003, the LLNL Director has lead the effort with his associate
directors, and a group of scientific and technical leaders to develop a
long range scientific and technology investment plan for the next 10
years. The plan has been developing in stages and LLNL has now
identified six thematic areas that cover the Laboratory missions and
core competencies. Preliminary plans have been developed by LLNL teams
of scientists and engineers. LLNL has made excellent progress and these
plans are scheduled for distribution in FY 2003. The six selected
thematic S&T planning areas are: (1) Stockpile Science and Technology;
(2) High-Energy Density Science and Technology; (3) Nuclear, Radiative,
and Astrophysical Science and Technology; (4) Biological, Chemical and
Materials Science and Technology; (5) Information Simulation and Systems
Science and Technology; and (6) Energy and Environmental Science and
Technology.

This will enable LLNL to balance their investments in their workforce,
foster interactions across disciplines and organization and facilities;
and position LLNL not only to continue to support the DOE/NNSA mission,
other federal agency missions such as Department of Homeland Security
and DoD, and in response to emerging national needs as well.

The LDRD Program is the key source of funding to support long term and
innovative research and development at LLNL. A mature review process is
in place at LLNL.

There are notable successes where LDRD was used as seed funding back in
1992 and 1993 so that new programs have been started at LLNL such as in
biodefense, the petawatt laser, and adaptive optics systems. FY 2003
projects are highlighted below:

-   Secure air-optic transport and routing network (SATRN) is a secure,
    high capacity communications networks to seamlessly route data
    between permanent fiber infrastructures including mobile platforms.
    LLNL has demonstrated its long-range, secure, air-optic lasercom
    links. This project is being transitioned to WFO funding

-   Bio-aerosol sensitive mass spectrometry (BAMS) is a real-time,
    reagentless single cell analysis technique. Unique molecular
    signatures have been demonstrated from a single bacterial spores
    thus allowing discrimination between bacteria and background
    aerosols. As a result of demonstration, the DoD has funded further
    development of this technology for biological aerosol detection.

-   Eyeglass was developed by LLNL scientists and engineers for
    fabricating large foldable Fresnel lenses for space deployment. LLNL
    has fabricated and tested a 5 meter diameter prototype. DoD and NASA
    are supporting further development of this technology under Work for
    Others.

During FY 2003, external Directorates Review Committees (DRC) bestowed
high praise to LDRD projects at LLNL. In addition to LLNL's summarized
comments other DRC comments regarding the quality of science and mission
relevance have been synopsized below on LDRD projects:

[Chemistry and Material Sciences Directorate Review Committee
Comments:]{.underline}

-   [Nanophotonics in Biosciences (CMS)]{.underline}. "This is an ideal
    topic for LDRD funding; it's fundamental, forefront science, and the
    people are first rate."

-   [Bio-aerosol Detection and Characterization (CMS)]{.underline}. "We
    have watched the

> development of this device for several years. Gard's performance has
> been excellent; he has produced a highly selective mass spectrometer
> that has been taken into the field successfully."

-   [Phonon Dispersion& Pu-Ga Alloys.(CMS)]{.underline} "This project
    led to exceptionally

> exciting results for the first measurements of complete phonon
> dispersion curves for a Pu alloy. These are truly impressive data and
> represent a contribution of exceptional merit and impact."

[Energy and Environment Directorate Review Committee
Comments:]{.underline}

-   [Performance Measure 5.1 (E&E).]{.underline} "...the LDRD projects
    presented to the Committee were of outstanding quality and were
    generally innovative."

-   [Stochastic Engine (E&E).]{.underline} "The committee was very
    impressed with the newly-developed Stochastic Engine (result of LDRD
    funding) and recognized it could form an important cornerstone for
    the new initiative in environmental consequences."

```{=html}
<!-- -->
```
-   [Overall E&E Directorate Grade]{.underline}. "....LDRD Investments
    are having significant results, and may are supportive of the new
    programs. The Directorate is successfully hiring talented staff, it
    is operationally competent, and it has a good safety record. Many of
    the projects reviewed were outstanding."

[Physics and Advanced Technology Directorate Review Committee
Comments]{.underline}:

-   [Executive Summary (PAT).]{.underline} "The DRC assessed a summary
    grade of "Outstanding/Excellent" for Measure 5.1; this grade
    reflects the belief that, while the individual LDRD projects are
    clearly outstanding, there is room for greater risk taking in the
    portfolio overall."

-   [Fourth Generation Light Sources (PAT).]{.underline} "...The work
    described included the development and preliminary testing of a
    novel reconstruction algorithm, and the development of tamping
    strategies to slow down the inevitable coulomb explosion. This is
    clearly an important and very innovative project that will prepare
    LLNL to be an early, successful user of the Linac Coherent Light
    Source."

-   [Ab initio modeling of Si Q-dots (PAT).]{.underline} "Galli's group
    at LLNL... has done some of the best work in the world in these
    areas..."

Review of the LLNL's external Directorates Review Committee reports are
indicative that LLNL has maintained an outstanding level in their
quality of science and mission relevance based on the projects validated
during this performance period. NNSA LSO concerns throughout the year
were addressed with LLNL and were taken care of promptly and
efficiently.

Concerns:

-   The upcoming retirement of C. K. Chou, Energy and Environment
    Associate Director, will be disruptive if the right replacement is
    not found in a timely manner.

  ---------------------------------------------------------- -----------------
  **Performance Measure 5.2**                                **Outstanding**

  **Nurture and maintain the Laboratories' signature         
  capabilities including unique experimental facilities and  
  competencies in support of Laboratory and external users   
  and sponsors**                                             
  ---------------------------------------------------------- -----------------

LLNL has maintained the Laboratories' signature capabilities including
unique experimental facilities and competencies and the overall rating
is at an ***outstanding*** level. However, there are several challenges
and they are addressed further in the evaluation for LLNL management to
take under advisement as part of continual quality improvements.

[Evaluation of LLNL's Experimental Capabilities.]{.underline} LLNL has
made a concerted effort in facility investments such as Supercomputing,
Laser Complex, IOAC, Forensics Science Center, Center for Mass
Spectrometry, Biosecurity and Nanoscience Laboratory, and Nuclear
Magnetic Resonance Center. Summary highlights are outlined below on a
few of these facilities. The external LLNL Directorate Review Committee
(DRC) reports singled out praise of LLNL's unique experimental
facilities/competencies in several areas:

1.  [Supercomputing]{.underline}. The Computation Directorate has
    contributed to the successful achievement of Performance Objectives
    5.1 and 5.3 as evidenced by its research and development program in
    software tools, advanced systems research, and computational
    mathematics. Objective 5.4 is supported by its effective
    partnerships with industry and academia which is leading this nation
    in the development of state of the art tools, computing systems and
    their collaborative efforts inn systems research. The Computation
    Directorate Review Committee stated, "The awards and honors received
    by the staff, society memberships, the publication record, and the
    numerous invited presentations all indicate superior technical
    vitality". "The Committee is unanimous in rating the Computation
    Directorate as outstanding in the quality of science and
    technology." This also applies to the DRC's rating for mission
    relevance and related objectives. "The high performance computing
    enterprise is essential to the core mission of the Laboratory. World
    class leadership in computation has consistently been achieved by
    the Directorate."

2.  [Building 174 (B-174) Laser Complex.]{.underline} The B-174 Laser
    facilities are essential for LLNL's institutional high energy
    density science program and to enhance and maintain the vitality of
    this laboratory in upgrading its signature experimental facility as
    part of its science and technology (S&T) base in Performance Based
    Measure 5.1. Research and development activities from nuclear
    physics to hydrodynamics are in process. S&T investments in high
    energy Petawatt science may contribute to radiography, astrophysics,
    and fast-ignition energy science. The Physics and Advanced
    Technologies Directorate Review Committee stated, "Overall, we
    assess this science program to be outstanding, in terms of both the
    quality of the work itself and its relevance to predictive
    capability within the weapons program." "As the only high-energy
    laser available for experiments on site, except for the future NIF,
    B-174 lasers should have an increasingly important role in both
    stand-alone experiments and as a test bed for NIF-bound experiments.
    In addition it is clear that, as a user facility, B-174 is a
    valuable resource for recruiting and training new scientists for
    LLNL's vast laser program."

3.  [Information Operations and Assurance Center (IOAC]{.underline}) is
    center of excellence at LLNL that develops knowledge discovery tools
    for the intelligence community sponsors, assesses foreign
    capabilities, and provides assistance in enhanced cyber security.
    IOAC met several milestones during this year; for example, an
    assessment of indigenous education and research and their ties to
    government organizations. NAI completed a cutting edge
    infrastructure advance in the development and installation of
    Livermore's Information Library and Analysis Center (LILAC) imagery
    processing archives. LILAC will be used for classified efforts.

4.  [Forensics Science Center (FSC).]{.underline} The FSC is a valuable
    center that houses LLNL's unique expertise and equipment to perform
    analytical chemistry skills. This center serves the needs not only
    for national security but for homeland security as well. FSC is now
    certified as an OPCW (Organization for the Prohibition of Chemical
    Weapons) laboratory for the analysis of collected samples during
    Chemical Weapons Convention inspections and is one of only two
    certified laboratories in the United States. Funding is received
    under a Work for Others Proposal with the Department of Commerce for
    ongoing work to support the OPCW work efforts. It has also received
    an ISO 17025 certification. The DRC stated, "FSC is a very
    sophisticated and specialized scientific service organization. As
    such, it derives strength from being embedded within CMS. It is an
    example of the type of unique, successful program that LLNL does
    well." "This gives the laboratory a unique qualification and it
    truly represents a "National Resource"".

5.  [Center for Accelerator Mass Spectrometry (CAMS).]{.underline} CAMS
    is a institutional facility which has broad science applications.
    CAMS runs over 10,000 research samples a year, serving multiple
    users such as LLNL, academia, and other federal agencies. CAMS
    provides unique capabilities in dosimetry of carcinogens and
    mutagens, oceanic and atmospheric chemistry, biogeochemical
    processes in carbon cycle, and detection of nuclear fuel
    reprocessing signatures and it services many small projects. CAMS
    has been recently made as part of the Water and Environment Program
    and part of the Environmental Sciences Division. The Water and
    Environment Program is a newly created program and is an attempt to
    expand on LLNL current environmental efforts aimed at
    characterization and remediation of contamination concerns related
    to water supply, availability, quality and concern with homeland
    security. The strategy is in process with major areas of technical
    focus identified.

> The Energy and Environment Directorate Review Committee stated, "In
> addition to its role as a world-class analytical capability for carbon
> age dating and numerous applications involving a wide spectrum of
> isotope determinations, CAMS serves as an NIH-funded National Resource
> for the application of isotopes to the fields of nutrition,
> toxicology, and pharmacology. Ultra-sensitive detection of isotopes by
> AMS is enabling studies of the fate and effects of low doses of
> chemicals in animal models and humans. This Resource, which is the
> only one of its kind in the U. S., is directed toward advancing AMS
> technology, training, university collaborators, and providing services
> to a broad user community."

6.  The [Energetic Materials Center]{.underline} is funded with 75 %
    core nuclear weapons funding and 25 % WFO. Chemistry and Material
    Sciences will operate the Energetic Materials Processing Center in
    the future. This center will process nearly all energetic materials
    for LLNL for three test sites. This program supports Department of
    Defense but also reaches out to major university faculty in order to
    understand chemistry at extreme conditions over a period of time by
    combining theory and experimental models to describe high energy
    materials performance; safety to understand damage mechanisms and to
    predict hazards and how to mitigate them; and surveillance which
    involves measurements and modeling in evaluating energetic material
    aging.

[Challenges:]{.underline}

[Future Challenges for LLNL.]{.underline}

[BSNL.]{.underline} The DRC report stated, "The Biosecurity Nanosciences
Laboratory (BSNL) is an exciting research program with some strong
scientific elements, well positioned to address the scientific issues
associated with pathogen detection. There is, however, insufficient
effort being devoted to overall device design and integration into a
sensor and information processing network; there currently exists no
clear consensus about architectural paradigms for integrated biosecurity
devices. This area needs to be strengthened in order for BSNL to have an
impact with its pathogen detection and collection approaches in the form
of workable small-scale devices and micro-device applications".

[CAMS]{.underline} is an outstanding unique experimental facility that
maintains a high productivity rate and maintains relevance to DOE
mission and national needs; however, CAMS needs to become more
self-supporting. The DRC states, "CAMS is not currently self-supporting.
It enjoys more than 50 % support from Laboratory funds." This support
could disappear if DOE/Laboratory needs to look at ways to reduce future
overhead costs.

[Water and Environment Program (E&E DRC Comment):]{.underline} The
Committee identified several key gaps that need to be addressed in order
to develop a robust and sustainable environmental consequences program.
(1) Data Collection and computing capability needs to addressed to
support this effort; (2) Lack of expertise at LLNL in the areas of
ecosystem biology and ecology. LLNL should look at recruiting expertise
in these areas.

  ---------------------------------------------------------- -----------------
  **Performance Measure 5.3**                                **Outstanding**

  **Strategically pursue and successfully execute a          
  portfolio of non-NNSA sponsored research that builds on    
  unique Laboratory capabilities and enhances the            
  Laboratories' competencies to meet current and future      
  national security needs**                                  
  ---------------------------------------------------------- -----------------

LLNL has strategically pursued and successfully executed a portfolio of
Non-DOE sponsored research that enhances the Labs competencies to meet
current and future national needs. Review of the LLNL's external
Directorates Review Committee reports are indicative that LLNL has
maintained an ***outstanding*** level in their quality of science and
mission relevance based on the projects validated during this
performance period with other federal agencies, academia and industry.
However, there is one concern on OCWRM and it is addressed further in
the evaluation for LLNL management to take under advisement as part of
continual quality improvements.

[Evaluation of LLNL Performance:]{.underline} In addition to the NNSA
programs, LLNL receives funding from many non-DOE agencies under the
Work for Others Program such as DARPA, National Institutes of Health,
NASA, other DoD agencies. LLNL strives to pursue a portfolio for Non-DOE
sponsored work that builds on unique laboratory capabilities, enhances
LLNL competencies to meet future national security needs. It is not
possible to describe all activities because the total level of effort in
this category is substantial. Summarized below are descriptions of
active ongoing successful projects. Work involving industry is
highlighted under performance measure 5.4.

[DOE Office of Science Projects Reviewed.]{.underline}

(1) LLNL receives about \$53 million in funding for research for the DOE
    Office of Science.

> The largest component comes from the [Office of Biology and
> Environmental Research (OBER)/]{.underline}Office of Science. The
> largest singe segment of the OBER work is LLNL participation in the
> Joint Genome Institute (JGI), a facility that is unique for research
> into the human genome. LLNL contributes outstanding leadership and
> technical capabilities to the Joint Genome Institute and Production
> Genomics Facility that successfully completed the sequence of three
> human chromosomes as part of the Human Genome Project and has had a
> major impact with its sequencing of a wide array of other organisms
> from microbes to the Popular tree. LLNL scientists also conducted
> state-of-the-art research using model organisms, such as the mouse, to
> understand the structure and function of the human genome. The DNA
> sequences of the puffer fish and sea quirt were both published in
> *Science* this year.

(2) LLNL collaboration on DIII-D (Tokamak Project) is one of the largest
    components of magnetic fusion research at LLNL. The quality of LLNL
    scientific work in these collaborations is outstanding. The LLNL
    staff, led by Steve Allen, has a significant leadership role in
    parts of this program. LLNL has published extensively and presented
    DIII-D results at major national and international conferences.
    Several LLNL scientists are actively involved in the International
    Tokamak Physics Activity. LLNL scientists continue to make
    contributions to the DIII-D research including Diverters, Advanced
    Tokamak, electron cyclotron heating, and diagnostics. LLNL's
    collaborative efforts on DIII-D bring in extensive effort and
    experience in developing and applying computer codes to edge physics
    issues.

(3) LLNL continues to provide outstanding progress in multiscale
    modeling of microstructural evolution and damage accumulation in
    structural materials that are highly relevant to fusion material
    research. The UEDGE (computer code) analysis performed by LLNL was
    critical to feasibility and attractiveness assessment of liquid
    surfaces. In the areas of plasma and fusion technologies, LLNL
    continued to play a lead role in edge plasma modeling for innovative
    plasma facing technologies based on flowing liquid surfaces.

(4) The achievement of the Sustained Speromak Plasma Experiment (SSPX)
    team in doubling the electron temperature and improving the energy
    confinement in speromak by nearly a factor of four is excellent.
    Substantial use of theoretical and computational modeling support by
    the team is an effective way in significantly advancing both the
    speromak and fusion science.

> In the IFE chamber engineering and technology research, there
> continues to be excellent work in areas of heavy ion chambers, final
> optics for laser drivers and focusing magnets for heavy ions and
> safety. Progress in developing a theoretical basis and simulation
> capability to provide a predictive end-to-end model of heavy ion
> fusion energy (IFE) driver system, and consideration of electron
> effects that are expected in high current experiment (HCE) at LBNL and
> the injector stand test at LLNL continues to be excellent. LLNL has
> provided excellent technical support and research for the IFE systems
> studies efforts of both dry wall and liquid wall concepts. The
> experimental data produced by collaborative research of LLNL, General
> Atomics, U. C. Davis and U. K. researchers on the transport of
> relativistic electron beams produced by petawatt lasers in high
> density materials is outstanding and will spur significantly new
> theoretical development in this area. LLNL has exhibited outstanding
> leadership within the IFE Technology community by keeping the IFE
> chamber program focused on critical issues. LLNL has conducted its own
> research activities within budget and on schedule. The IFE work is of
> the highest quality and is disseminated well through publication in
> journals and at technical conferences, for example, at the recent
> Third International Conference in Inertial Fusion Science and
> Applications 2003. The IFE engineering and technology work at LLNL is
> clearly in support of national needs and is directed toward future
> energy requirements as stated in the National Energy Policy.

(5) Livermore staff has made numerous modeling improvements to their
    global atmospheric chemistry model and have applied their model very
    effectively in support of DOE's Atmospheric Science Program (ASP).
    Data from several recent major ASP field campaigns have been
    analyzed, and their chemistry model has been effectively applied to
    a wide range of air quality, climate, and homeland security
    considerations.

> LLNL PCMDI is a unique, critical organization that supports well the
> Department's climate modeling program and provides excellent and
> important service to the entire research community.
>
> Research conducted in the Ocean Carbon Sequestration Research Program
> has been highly relevant and of excellent scientific merit. LLNL
> research continues to be published in journals and has presented
> results in important national and international scientific meetings.
>
> \(6\) LLNL research supported by the Materials and Engineering Physics
> program is excellent. It is relevant to national needs and the agency
> mission. The project on the [Kinetics of Phase Transformations in
> Welds]{.underline} involves modeling of reactions in weld heat
> affected zones by means of real-time synchrotron experiments and was
> found to be world class state-of-the-art research that is providing
> new insights into reactions and mechanisms during welding.
>
> Peer review of the project [Solid State Physics/Advanced Positron
> Materials Characterization]{.underline} indicated it to be of
> exceptional quality, with experimental facilities that are "second to
> non" and theoretical work that is "world leading".

[Notable Work For Others Projects at LLNL were Reviewed during
2003]{.underline}:

[National Institutes of Health]{.underline} (NIH) is funding several
projects related to the causes of dietary cancer, DNA repair and
development of healthcare applications. A long term focus of the NIH is
investigating the food mutagens causing cancer in humans. The Biological
and Biotechnology Research Group external Review committee stated that
the DNA repair work done by Larry Thompson's group is "as good as it
gets and would be first rate at any top U. S. university on the basis of
scientific quality and impact".

The Biology and Biotechnology Research Group at LLNL support the Office
of Science's JGI and its members support a variety of projects such as
pathogen DNA and protein signature development, high-throughput DNA
sequencing, whole-genome analysis, as well as funding received from a
WFO sponsor, Integrated Molecular Analysis of Genes and their Expression
([I.M.A.G.E) Consortium]{.underline}, which is jointly funded by NIH and
OBER. The consortium started in 1993 with the goal of developing
complimentary DNA resources to advance the new gene discoveries. The
consortium represents the largest public collection of cDNA library
collection, analysis and maintenance, comparative genomics, mapping, and
protein structure efforts.

[NASA]{.underline} supports several projects at LLNL that are an
integral part of LLNL efforts in astrophysics. LLNL IMPACT code forms an
important component of the NASA Global Modeling Initiative to analysis
of x-ray spectroscopy of astrophysical plasmas.

LLNL supports over 100 projects for [Department of Defense]{.underline}
(DoD) programs. Most projects constitute technology development and
applications efforts where results benefit both DoD and support DOE
missions.

[Department of Defense/Secret Service - Counter Proliferation Analysis
Planning System (CAPS)]{.underline} is a tool used by the U. S. military
to analyze proliferator weapons production capabilities, provides
interdiction options and consequences. During March and April 2003, LLNL
personnel supporting CAPS provided 24/7 on call support to DoD during
Operation Iraqi Freedom.

[Department of Defense - Joint Conflict and Tactical Simulation
(JCATS)]{.underline} is a multi dimensional joint conflict simulation
tool that provides simulation capabilities to the DoD with communication
and computing links across ground, air, and water. This tool is now used
by multiple sponsors including all four military services, the Secret
Service, for planning, training, and counterterrorism maneuvers.

DoD's sponsored project on conflict simulation modeling is being
extended to take on the homeland security mission challenge by [ACATS
(Analytic Conflict and Tactical Simulation]{.underline}) that will
facilitate the evaluation of emergency response in dealing with
chemical, biological, or nuclear attacks in an urban environment.

[National Guard - Homeland Operating Planning Systems
(HOPS)]{.underline} is under development at LLNL to assess the
vulnerability of potential target facilities, evaluation of various
emergency response plans, characterization of an attack and collateral
damage, and how to manage a response. This project is supporting
National Guard exercises during FY 2003.

The Forensic Science Center (FSC) was established as a Q-cleared
analytical laboratory in support of Stockpile Stewardship Program.
However, other work for other sponsors such as the FBI and DoD also
support activities in the center for collection and analysis of forensic
samples with chain of custody/control. The FSC has now been certified as
an OPCW laboratory and LLNL has received its ISO 17025 certification.
WFO sponsor Department of Commerce has been providing funding in
connection with LLNL's support of analytical sample for the
[Organization of Prohibition of Chemical Weapons]{.underline} (OPCW).
This center is now well-positioned to service the needs for the
Department of Homeland Security.

[DOE Office of Civilian Radioactive Waste Management (OCRWM)
Project]{.underline}:

LLNL has been involved in the Yucca Mountain Project since 1977 and the
current focus is on developing a system of engineered barriers that is
surrounded by natural barriers to contain the radioactive waste. During
this year, LLNL performed materials testing and performance modeling for
the storage canister and the engineered barrier systems.

While the quality of LLNL's work in the areas of NWTRB interactions,
Long-Term Corrosion Test Facility, and engineered barrier systems (EBS)
have been excellent, other issues were taken into account to arrive at a
rating of excellent for OCRWM.

LLNL continues to maintain the one-of-a-kind Long-Term Corrosion Test
Facility, data from which supports the extrapolation to 10,000 years of
the general corrosion rates for alloy-22 and Ti. Additional samples in
this facility provide long term data on crevice corrosion and
stress-corrosion cracking. The laboratory has developed short-term
electrochemical corrosion testing to define initiation of localized
corrosion in expected repository environments. The laboratory has
developed and initiated testing to create T vs. Rh curves defining
deliquescence for dust as well as seepage water mineral assemblages.
Work is documented in project analyses model reports.

LLNL has provided excellent support to OCRWM in their interactions with
the NWTRB. Informal interactions with Board members and staff in January
2003 at the LLNL facilities were very productive and informative for the
Board. LLNL staff also provided excellent support to the formal NWTRB
meetings in January, May, and September as either presenters or support
for presentations on in-drift environment and performance confirmation.

LLNL's work on ORD's EBS Multi-Scale Thermal Hydrologic (MSTH) Model is
quite good and its quality notable. They have been responsive to making
the changes to the documentation of this model based upon peer review
comments as well.

LLNL participated in the Frontline Leadership Program, which
demonstrated support of the cultural changes at Bechtel SAIC (OCRWM's
M&O contractor).

[DOE HQ Concerns.]{.underline}

[OCRWM Concern]{.underline}. LLNL delayed the closure of deficiencies in
scientific notebooks. Their planning/cost control was inadequate in the
area of waste package/drip shield testing. Cost overruns and schedule
delay resulted.

LLNL response time for EBS work could have been more-timely, and their
ability to communicate and integrate their technical work with other
program participants could be improved. For example, the comments on the
MSTH model that required follow-on work could have been avoided to some
extent had LLNL provided a greater degree of integration. Overall,
improvement is needed in the integration of various testing activities
within the testing group.

Responsiveness to BSC Management has improved during the past few months
thanks to the new management team at LLNL. One example of this is the
support provided by LLNL management and staff towards completion of the
Technical Basis Document on Waste Package and Drip Shield Performance.

This concern should be reviewed by LLNL Management to ensure continuous
quality improvements are made with the DOE OCRWM organization during FY
2004.

[Office of Science Concerns:]{.underline}

\(1\) Although the pelletron positron facility at LLNL is not a "major"
research facility, its performance and output is a great concern to BES
and the Materials and Engineering Physics program.

\(2\) The Condensed Matter Physics program terminated the intense
accelerator positron source project at LLNL for lack of progress. It is
regrettable that this advanced positron source was not brought in and
made to function. LLNL could not find the right people to put this
effort together.

  ---------------------------------------------------------- -----------------
  **Performance Measure 5.4**                                **Outstanding**

  **Foster active participation in the broad scientific      
  community and leverage unique Laboratory expertise and     
  capabilities to develop strategic collaborations with      
  other national laboratories, industry and academia.**      
  ---------------------------------------------------------- -----------------

Lawrence Livermore National Laboratory (LLNL) has actively participated
in the broad scientific community and has leveraged unique Lab expertise
and capabilities to collaborate with other labs, industry, and academia.
Review of the LLNL technology transfer projects such as Cooperative
Research and Development Agreements (CRADAs) and licenses during this
performance period are indicative that LLNL has maintained an
***outstanding*** level in their quality of science and mission
relevance based on the projects validated during this performance period
with industrial partners. Awards and recognitions are an important
measure and are described below. Moreover, the success of DOE/LLNL's
industrial partners to bring products to the commercial marketplace is
even more noteworthy.

**[DOE Mission Relevance and National Needs]{.underline}**. LLNL engages
in technology transfer in order to:

(1) Support DOE/NNSA national security and other missions and programs;

(2) Promote U.S. economic competitiveness in world markets;

(3) Promote economic development in the local area and the entire U.S.,
    with an emphasis on small business; and,

(4) Enhance the quality of life for all Americans.

Partnership examples are highlighted below that show support of the
DOE/NNSA's mission:

-   LLNL signed a non-exclusive license for the Handheld Advanced
    Nucleic Acid Analyzer (HANAA) technology, expected to be useful for
    chemical and biological detection, with Smiths Detection --
    Edgewood, Inc. (formerly Environmental Technologies Group, Inc.
    (ETG) and now a subsidiary of Smiths Aerospace). The HANAA
    technology is at the heart of Smiths Detection Bio-SeeqTM product,
    currently being marketed as the first portable, hand-held
    thermocycler capable of detecting both bacterial and viral
    pathogens. Smiths Detection's objective for the license is to
    provide the DoD and the intelligence agencies with highly portable,
    advanced, bio-detection instruments and to further the DOE objective
    of putting advanced instrumentation for the detection of biological
    terrorist agents into the hands of first responders. The HANAA
    analyzes biological samples for the presence of specific DNA
    sequences that serve as the fingerprints of specific pathogens. It
    can simultaneously test four samples, and each sample for two
    different DNA sequences. The results are reported in about 20
    minutes. The HANAA provides the first handheld, portable,
    field-worthy, real-time PCR bio-detection instrument. It is ideally
    suited for emergency response where biological pathogens are
    suspected, and for field monitoring where portability and fast
    answers are critical (e.g., monitoring water or food supplies for
    biological contamination in real time). It can also be used in
    intelligence, combat or reconnaissance missions.

-   LLNL has developed a lightweight portable Gas
    Chromatograph-Quadrupole Mass Spectrometer (GC-QMS) for use by
    organizations to detect and identify hazardous substances related to
    terrorist attacks, demilitarization, and law enforcement with the
    support of the Technical Support Working Group (TSWG). The TSWG is
    the U.S. national forum that identifies, prioritizes, and
    coordinates interagency and international research and development
    requirements for combating terrorism and supports rapid development
    of the required technology. Current devices available on the market
    are either too cumbersome or do not detect and identify all the
    necessary substances. Both Constellation Technology Corporation and
    Kobe Steel, Ltd. licensed the GC-QMS technology. Constellation is a
    small business located in Largo, Florida with expertise in
    commercializing radiation detection and analytical instrumentation.
    Kobe Steel, a major Japanese corporation with interest in supporting
    the Japanese chemical weapons disposal program, has a license
    restricted to manufacture in the United States.

-   LLNL signed a license with Spectrolab Inc. for the Silicon
    Monolithic Microchannel (SiMM) Cooled Laser Diode Array technology
    in January 2003. The SiMM Laser Diode Array is a high-performance
    package for laser diode bars. It combines efficient cooling with
    precision diode-bar optical conditioning. This technology won an R&D
    100 Award in 2002. Spectrolab Inc. is a subsidiary of The Boeing
    Company located in Sylmar, California and is the world's leading
    manufacturer of space solar cells and panels. Spectrolab Inc. plans
    to use the licensed technology to expand its capability in producing
    products to meet the optical and laser needs of industry and the
    government.

-   The **Laser Shot Peening** technology developed under a CRADA
    between LLNL and Metal Improvement Company has successfully spun out
    to industry and in the first 14 months of commercial production has
    already improved the safety and reliability as well as dramatically
    reduced the maintenance costs of over \$20B in commercial aircraft.
    The technology is a winner of the 2003 R&D 100 Awards competition.
    LLNL is now working to spin back this proven technology into
    government and military applications.

```{=html}
<!-- -->
```
-   LLNL executed a **United States Industry Coalition (USIC) CRADA with
    BioLuminate, Inc**. as part of the Initiative for Proliferation
    Prevention (IPP). The project, \"Breast Cancer Diagnostic System:
    Data Analysis and Algorithm Development,\" is a collaborative effort
    to develop algorithms for BioLuminate\'s breast cancer diagnostic
    system that can process data to provide physicians with an immediate
    diagnosis without the need for the conventional biopsy. LLNL,
    BioLuminate, and BioFil (a private company located in Russia) are
    working together to analyze data acquired with the BioLuminate
    Breast Cancer Diagnostic System in clinical trials. BioFil has
    developed various medical devices and has the scientific and
    analytic capabilities to significantly accelerate the development of
    the breast cancer diagnostic system. This NNSA sponsored project
    will also advance the non-proliferation goals of the U.S.
    government. As part of the IPP program, this project will engage
    former Soviet Union weapons personnel in R&D and commercial pursuits
    that may lead to stable, civilian employment opportunities, thus
    reducing the risk of proliferation.

-   LLNL and **Lynntech, Inc.** executed a CRADA for the development of
    a DNA/RNA Sequencer for Rapid Assessment of Infectious Agents and
    can be used for homeland security to counter terrorism or for
    medical purposes. The objective of this project is to design,
    fabricate and test a prototype of a new portable microfluidic
    instrument for the sequencing of a single strand of DNA/RNA.
    Lynntech and LLNL will construct and demonstrate an alpha prototype
    unit containing all the necessary subsystems including sampling and
    sample handling, fluid handling, electrochemical detector and output
    signal processing.

> []{.mark}

-   **Zyomyx**, **Inc**. executed a CRADA with LLNL to develop a system
    capable of over-expressing and purifying thousands of human proteins
    in an arrayed, addressable format in quantities and purities
    suitable for high-throughput functional studies. Zyomyx, Inc.
    specializes in the development of new antibodies and high-density,
    antibody-based biochips as diagnostic products. The work to be
    performed under this CRADA will be funded in part by a
    Work-for-Others (WFO) project between LLNL and the UC Discovery
    Grant Program (UC BioSTAR Project), which funds projects involving a
    California company and a UC campus or UC-operated federal lab. UC
    BioSTAR is a matching funds program that requires the company to
    fund one half of the work at the UC campus or federal lab with the
    other half of the funding provided directly by UC BioSTAR

-   LLNL executed a CRADA with **Cummins Engine Company** for the
    Homogeneous Charge Compression Ignition (HCCI) Combustion Engine
    technology. HCCI is an alternative piston-engine combustion process
    that can provide efficiencies as high as compression-ignition,
    direct-injection (CIDI) engines (an advanced version of the commonly
    known diesel engine) while, unlike CIDI engines, producing ultra-low
    oxides of nitrogen and particulate matter (PM) emissions. The
    purpose of this CRADA is to advance the state of the art on HCCI
    engines, resulting in a clean, high-efficiency alternative to diesel
    engines thereby reducing our dependence on foreign oil.

[]{.mark}

> **[Outreach Activities]{.underline}**. Outreach is essential in
> furthering technology transfer. LLNL has provided space to the
> **Tri-Valley Technology Enterprise Center** (TTEC) to establish an
> on-site resource center for small businesses and start-up companies
> from the local area, including any that originate from LLNL, SNL or
> that partner with the labs. The TTEC was established through a
> collaborative effort of the city of Livermore, the Tri-Valley Business
> Council, the labs, and local businesses. It is aimed at promoting the
> economic vitality of the local region and helping facilitate a bridge
> between the laboratories and the local technology community. TTEC
> currently has five tenants on-site at the LLNL facility and two
> tenants at an associated off-site facility. TTEC also serves the local
> business community by providing educational seminars open to the
> public. These have covered topics such as the protection of
> intellectual property rights, sources of funding, success factors for
> startup companies.

[]{.mark}

> During FY 2003, **Technology Ventures Corporation** (TVC) opened an
> office in Livermore, CA. Since 1993, Technology Ventures Corporation
> has been an important contributor to the formation of new businesses
> built on leading-edge technologies developed at NNSA laboratories and
> in the expansion of existing businesses. TVC assists client companies
> to position themselves to obtain investor interest and funding. TVC
> identifies technologies with commercial potential, coordinates the
> development of business and management capabilities and seeks sources
> of capital investment for the business. TVC is not a funding
> institution, but a bridge between technology and investment.

**[Academia Collaborations]{.underline}**. LLNL has many successful
partnering agreements with industry and collaborations with other
laboratories and academia. Two academia collaborations are highlighted
below:

-   **[National Cancer Institute at U. C. Davis.]{.underline}** The UC
    Davis Cancer Center received National Cancer Institute Designation.
    BBRP's DRC stated that this achievement is "a good example of how
    LLNL investigators can utilize their unique resources and strengths
    in chemistry and carcinogenesis....to achieve research excellence".

-   **[Biophotonics Center at U. C. Davis]{.underline}**. This center
    was recently established at U. C. Davis and is funded by the
    National Science Foundation to under the working of cells and
    tissues in living organisms. LLNL collaborates with UC Davis and
    other academia such as UC Berkeley, UC San Francisco, Stanford
    University, University of Texas, Louisiana state University and
    others.

**[Technology Transfer Awards.]{.underline}** LLNL arranges partnerships
with industry by utilizing CRADAs, licensing agreements, Work for Others
(WFO) Agreements and procurements for research and development (R&D).
WFO agreements were highlighted under performance measure 5.2.

LLNL was recognized for its high quality of research and development
technology transfer activities whereby they received six 2003 **[R&D 100
awards]{.underline}** and one special **[R&D Magazine Editor's Choice
award]{.underline}**. Since 1978, LLNL researchers have garnered 97 R&D
100 awards. The Laboratory\'s six plaques in 2003 represent the second
consecutive year that Livermore employees have won six R&D 100s. R&D
Magazine presents the awards annually for the top 100 technological
achievements that promise to improve people's lives through breakthrough
products and processes. Winning entries are selected on criteria that
include evidence of commercial viability. Following is a brief
description of each of this year's LLNL winners:

-   [Biological Aerosol Sentry and Information System
    (BASIS).]{.underline} In an award shared by Los Alamos and Lawrence
    Livermore, this advance permits the early detection of biological
    pathogens. The BASIS technology was deployed at the 2002 Winter
    Olympic Games in Salt Lake City, Utah.

-   [Extreme Ultraviolet Lithography (EUVL) Full-Field Step-Scan
    System]{.underline}. The EUVL system is a fully integrated system
    that prints 50-nanometer (billionths of a meter) features on
    computer chips, almost twice as small as features possible with
    other systems. The award has been given jointly to researchers from
    Lawrence Livermore, Lawrence Berkeley, and Sandia national
    laboratories, as well as Northrop Grumman Space Technology/Cutting
    Edge Optronics.

-   [Ion Beam Thin-Film Planarization Process.]{.underline} This
    LLNL-developed process smoothes defects and roughness on reflective
    mask blanks and has helped solve a technical challenge for producing
    faster computer chips with more memory by using EUVL. The process
    will be an integral part of next-generation EUVL systems.

-   [High-Average-Power Electro-Optic Q Switch]{.underline}. This
    technology allows optical switching of high-average power lasers for
    energy research, machining, and national defense. The Q switch
    offers a ten-fold increase in the average power handling capability
    for lasers from 30 watts to 300 watts.

-   [Lasershot Precision Metal Forming System]{.underline}. This
    technology is a revolutionary approach to shaping large-panel
    structural components, such as in the aviation industry. This award
    is shared with Metal Improvement Company, Inc.

-   [MEMS-Based Adaptive Optics Phoropter (MAOP).]{.underline} The MAOP
    technology combines advances in astronomy and micromachining to
    enhance vision and improve early diagnosis and treatment of retinal
    diseases. The award is shared by LLNL, Sandia National Laboratories,
    the University of Rochester, Wavefront Sciences, Boston
    Micromachines Corp., and Bausch & Lomb.

-   The [Extreme Ultraviolet Lithography (EUVL) Full-Field Step-Scan
    System]{.underline} was also singled out for special recognition at
    the R&D 100 Awards Banquet in Chicago. The EUVL team members from
    the Virtual National Laboratory (LLNL, Sandia, and LBNL) received
    the Editor's Choice award for "Greatest Improvement Upon An Existing
    Technology." This special honor recognizes the broad scope and
    significance of the EUVL project's achievements. The award, one of
    three special recognition awards, was presented to only three of the
    one hundred technologies that, in the opinion of the editors of R&D
    Magazine, represented the most outstanding achievements among this
    distinguished group.

[]{.mark}

The **[Federal Laboratory Consortium]{.underline}** (FLC) has recognized
LLNL for its "excellence in technology transfer." In 2003, LLNL shared
an award with Lawrence Berkeley and Sandia, and received three honorable
mentions for other LLNL nominations. The FLC is a nationwide network of
federal laboratories that provides a forum to develop strategies and
opportunities for linking laboratory mission technologies and expertise
with the marketplace. The 2003 FLC Awards for Excellence in Technology
Transfer recognize laboratory employees who have accomplished
outstanding work in the process of transferring a technology developed
by a federal laboratory. This year's honors went to:

[]{.mark}

-   [Extreme Ultraviolet Lithography Tool]{.underline}. EUVL technology
    tops the charts again with its next-generation lithography
    techniques that result in the ability for microprocessors to be made
    10 times faster, with active transistors and memory chips that can
    store 40 times more information. The award has been given jointly to
    the team from Lawrence Berkeley, Lawrence Livermore, and Sandia
    national laboratories -- collaborating as the Virtual National
    Laboratory. EUVL technology and its associated knowledge have been
    transferred under a CRADA, which will result in the technology
    transitioning into commercialization.

The FLC also recognized LLNL with "honorable mentions" for the following
contributions to technology transfer.

-   [Field-Portable Thin-Layer Chromatography (TLC).]{.underline} LLNL
    developed the field-deployable TLC unit in the late 1990s for the
    U.S. Army, Defense Ammunition Center, for on-site detection of the
    level of stabilizer in the propellant of ammunition and other
    weapons. In 2001, Livermore licensed the technology to ALU LIKE
    Enterprises, LLC (ALE), which has established a plant in Hilo, a
    designated U.S. Historically Underutilized Business (HUB) zone.
    Producing the TLC field kits in Hawaii seemed a natural, given the
    large inventory of demilitarized and active ammunition located there
    and throughout the Pacific. Their new Hilo plant, dedicated in April
    2002, is participating in the Federal Experimental Program to
    Stimulate Competitive Research (EPSCOR).

-   [The Network Intrusion Detector (NID).]{.underline} The
    LLNL-developed NID technology is a computer security system
    consisting of a suite of tools that detect and analyze unauthorized
    computer access, to assure the security of computer systems engaged
    in sensitive work at LLNL, the Department of Energy, and Department
    of Defense. LLNL transferred the NID technology to WareOnEarth
    Communications, Inc., an established organization providing
    information assurance services to both the private and public
    sectors, after undergoing a considerable set of actions to protect
    federal security interests and clarify the intellectual property
    issues of a code that consists of both open and proprietary
    elements.

-   [Field-Portable Kits for Solid-Phase Micro-Extraction Technology
    (SPME).]{.underline} Solid-phase microextraction safely and
    efficiently collects and samples organic volatile materials using
    optical fibers as "chemical dipsticks." LLNL's Forensic Science
    Center developed two devices to hold the extremely fragile optical
    fibers, one for hazmat first-responder sample collection and the
    other for sampling applications in which more samples and data will
    be analyzed and hand-carried back to a laboratory. LLNL signed a
    license in 2002 with Field Forensics, Inc. for two field portable
    SPME kits and a SPME fiber reconditioning unit. The company started
    manufacturing kits and accepting orders in May 2002, and intends to
    support U.S. military demilitarization and characterization
    operations and emergency first responder applications in which
    careful handling of very hazardous samples is required, as well as
    agencies involved in chemical warfare agent counter-proliferation
    and anti-terrorist activities.

Following are examples of LLNL technology transferred successfully
through licensing agreements:

[]{.mark}

> \(1\) **[ORTEC]{.underline}** Products, a business unit of AMETEK,
> signed a non-exclusive license agreement with LLNL in April 2003 to
> commercialize **RadScout,** a radiation detector and analyzer. A
> special signing of the licensing agreement took place during a
> ceremony attended by National Nuclear Security Administrator Linton
> Brooks, who stated, "RadScout is an excellent example of NNSA
> laboratories providing solutions to help our nation improve homeland
> security and assist in the war on terrorism." The detector features a
> miniaturized refrigeration system that eliminates the need to carry
> liquid nitrogen to cool the device's high-purity germanium crystals.
> Those crystals are used to detect minute amounts of neutrons and gamma
> rays emitted by radioactive materials.
>
> First responders could use these high-performance, high-resolution
> portable systems at border crossings, cargo ship docks, transportation
> terminals, post offices, etc. to quickly differentiate between
> potentially dangerous radioactive materials and harmless radiation
> sources, and to determine whether or not they pose a threat. ORTEC
> plans to market the detector as the Detective and Detective-EX.
>
> \(2\) **[BAE Systems]{.underline}** signed a non-exclusive license
> with LLNL to develop, manufacture and sell the **Integrated Optical
> Capillary Electrophoresis (IOCE) Chemical Microsensor** technology.
> This technology is useful for chemical and biological detection. The
> sensor, which depends on innovative laser technology developed at
> LLNL, is the size of a suitcase. BAE and LLNL are planning to
> collaborate on the miniaturization of the sensor. BAE Systems markets
> a wide range of products such as electronic warfare systems, space
> electronics, reconnaissance, surveillance, avionics, and navigation
> equipment.
>
> \(3\) **[General Atomics]{.underline}** signed a license with LLNL for
> **Multilayer Dielectric Diffraction Gratings** technology. The
> Multilayer Dielectric Diffraction Grating is for developing
> multi-wavelength laser systems such as those used in
> telecommunications systems with short pulse and pulse compressed
> lasers. General Atomics specializes in diversified research and
> development in energy, defense, and other advanced technologies, many
> for environmentally sensitive applications.
>
> \(4\) **[General Atomics]{.underline}** signed another license with
> LLNL for **Inductrack** technology, a magnetic levitation system,
> using high-field permanent magnets to create its own levitating fields
> for urban and high-speed maglev train systems. The features include
> passive levitation, lower cost and maintenance requirements compared
> to those of existing maglev systems, tight turn radius, steep
> hill-climbing capability, and low noise. General Atomics is the prime
> contractor for the General Atomics Low Speed Maglev Technology
> Development Project, one of the projects funded by the Federal Transit
> Administration (FTA) as part of their Urban Maglev Program. The
> overall FTA program objective is to develop magnetic levitation
> technology as a cost effective, reliable, and environmentally sound
> transit option for urban mass transportation in the United States.

**[Program Planning and Performance]{.underline}**. In FY 2003, the
Laboratory has executed 63 patent and copyright licenses, and 4 license
option agreements. Licensing revenue is \$4.1 million, the highest ever.

The Laboratory has executed 10 new CRADAs and 8 amendments, with 29
CRADAs currently active. The proportion of CRADA activity funded by
private industry was 71%, at \$4.6 million, with the remaining 29%, or
\$1.9 million, funded by the IPP program and various DOE programs
outside NNSA. There were 163 inventions reported, 109 U.S. patent
applications filed, and 37 initial foreign patent applications filed in
FY 2003. A total of 67 U.S. patents and 19 foreign patents were issued
in FY 2003 for Laboratory inventions.

LLNL has maintained mature policies, procedures, financial databases,
and processes. LLNL has maintained a good working relationship with NNSA
and its industrial partners throughout FY 2003 and is well deserving of
its outstanding rating.

  ---------------------------------------------------------- -----------------
  **Performance Objective 6**                                **Outstanding**

  **Achieve successful completion of projects and            
  development of user facilities**                           
  ---------------------------------------------------------- -----------------

  ---------------------------------------------------------- -----------------
  **Performance Measure 6.1**                                **Outstanding**

  **Execute significant construction projects as identified  
  and agreed to between the Site Offices and laboratories    
  within budget, scope and schedule**                        
  ---------------------------------------------------------- -----------------

The Laboratory did an **outstanding** job in achieving the FY 2003 goals
and far exceeded the performance indicator; "Measures and evaluation for
ETCU, SCIF and TSF employing earned value performance measurement
techniques. Cumulative cost variance and cumulative schedule variance
should be less than 10% negative for the year." Please note that:

ETCU = Engineering Technology Complex Upgrade; Line item project number
02-D-105

SCIF = Sensitive Compartmented Information Facility; Line item project
number 01-D-800

TSF = Terascale Simulation Facility; Line item project number 00-D-103

**[Evaluation of Performance for ETCU, SCIF and TSF]{.underline}**

Measures and evaluation for ETCU, SCIF and TSF employing earned value
performance measurement techniques: The cumulative cost variance and
cumulative schedule variance is 0.997 through September 2003. This
cumulative variance is significantly negatively impacted by budget
actions taken by the program affecting the ETCU project which were
beyond the control of the project group. The real schedule performance
of the ETCU was much better than the .86 being reported. Please see the
ETCU project write-up below regarding the decisions by LLNL to deal with
the FY 2003 budget rescission and their effect on the project funding.
Therefore the actual cumulative performance is really about 1.02, but
the corrective documentation for ETCU will not be in place until the
November-December 2003 time frame. This actual average substantially
exceeds the objective of .90 performance by project as of September 2003
is as follows:

  ----------------------- ----------------------- -----------------------
  Project                 Cost Performance Index  Schedule Performance
                          (CPI)                   Index (SPI)

  TSF                     1.02                    1.07

  ETCU                    1.06                    0.86

  SCIF                    1.01                    0.96
  ----------------------- ----------------------- -----------------------

> The TSF project is proceeding within budget and ahead of schedule. The
> project was able to weather a \$1M cost reduction resulting from the
> rescission and general reduction during this reporting period. It is
> anticipated that this will be a very successful project that will
> complete ahead of schedule.
>
> The ETCU project team during a FY 2003 first quarter project review
> discovered the need to increase the preliminary baseline budget for
> the seismic upgrade subtask by \$11M. After reviewing three recovery
> options with Lab Management and NNSA, the option of maintaining the
> original budget through selected scope reduction was chosen. An
> Independent Review was conducted, the entire project was examined and
> the review team validated the selected option. LSO and NNSA
> Headquarters approved a Baseline Change Proposal documenting the
> revised scope and establishing a revised project baseline. This was a
> very good example of the how the "planning and engineering design
> (PED) process" is supposed to work. David Crandall, NA-11,
> complemented the LLNL and NNSA teamwork that handled this situation by
> calling it a model for others to follow.
>
> This past spring during rescission and general reduction negotiations,
> a decision was made to withhold \$5.4M of FY 2003 funding from the
> ETCU project and to restore the funding in FY'05. This will extend the
> schedule and delay completion of the project. This decision was beyond
> control of the project team and no additional funds will be
> forthcoming to compensate for the delay. The current schedule baseline
> has been unattainable due to this change in funding even though the
> currently funded work is on schedule. The project is currently being
> reported as behind schedule due to this change and not having a
> revised baseline. A recovery plan has been prepared and a Baseline
> Change Proposal will be processed to incorporate the revised plan
> after a favorable review from an Independent External Review team in
> October 2003.
>
> The SCIF project identified cost issues in the Spring time frame. A
> project manager change was made and the new PM did a complete review
> of the project which resulted in the confirmation that there was
> insufficient contingency to complete the project. An aggressive
> corrective action plan was developed immediately to reallocate certain
> costs to appropriate cost centers and to delete some scope which would
> not compromise the functionality of the facility. This resulted in the
> recovery of \$800K for contingency which is believed to be quite
> adequate to complete the project. The corrective action plan was
> reviewed and accepted by Lab management, LSO, and NNSA headquarters.
> Baseline Change Proposals documenting the results of the corrective
> actions and reestablishing the project baseline were approved by LSO.
> Lessons Learned from this series of events are being thoroughly
> documented and disseminated. The project is on track to complete the
> construction phase in December 2003 within budget and on schedule. The
> accreditation phase will follow and project completion is scheduled
> for November 2004.

**[NIF]{.underline}**

The Laboratory did an **[Outstanding]{.underline}** job on execution of
the NIF Project in accordance with plans for FY 2003. All 36 DOE/NNSA
performance milestones scheduled for completion in FY 2003 were
completed on or ahead of the DOE/NNSA target dates. In addition, several
DOE/NNSA milestones scheduled for completion in FY 2004 were completed
ahead of schedule in FY 2003. Five DOE/NNSA level 2 milestones were
completed ahead of schedule, including "First 1ω Light" and "First 3ω
Light to the Target Chamber Center". Major accomplishments included
completion of the laser beampath enclosures, installation of the first
target area diagnostics, and completion of activation and commissioning
of the four NIF Early Light laser beams.

During FY 2003 the earned value schedule variance stayed between -\$16.3
million and -\$3.2 million, with a corresponding schedule performance
index between 0.99 and 1.00. The earned value cost variance stayed
between -\$17.2 million and +3.9 million, with a corresponding cost
performance index between 0.99 and 1.00.

NIF Project safety continues to be an area of exemplary achievement,
with more than 3.4 million construction person-hours having been worked
without a lost time injury. In addition, the 12-month total recordable
case rate at the end of FY 2003 was 1.5, which remains well under the
National and California averages.

During FY 2003 NIF Project Office communications met NNSA expectations
and there were no surprises. All NIF monthly reports were submitted to
the NNSA Office of the NIF on schedule.

The April 2003 DOE Inspector General audit report on the "Status of the
National Ignition Facility Project" and the UC FY 2003 Status Review of
NIF concluded that the NIF Project is meeting cost, schedule, and
technical baselines.

  ---------------------------------------------------------- -----------------
  **Performance Measure 6.2**                                **Outstanding**

  **Develop with NNSA and implement a National Hydrotest     
  Plan that addresses mutual utilization of hydrotest        
  facilities, including containment and materials            
  availability**                                             
  ---------------------------------------------------------- -----------------

LLNL has managed its own hydrotest program well, as evaluated elsewhere,
and during this period was well along in the preparation of a 2nd
experiment at the DARHT facility. There are, however, significant
shortfalls in the development of an integrated plan by the two
University of California Laboratories. NNSA has not been a party to the
UC Division Review Committee process and has not been actively included
in discussions regarding the formulation of the plan to insure that it
addressed the issues of interest to NNSA, namely understanding the
hydrotest requirements, priorities for shots and ensuring that national
facilities are best utilized to execute a national plan. In particular
the joint planning has not resulted in any concrete planning to execute
LANL experiments at LLNL facilities (NA-113 Van Fleet)

LLNL has effectively managed the mitigation issues associated with
hydrotests. Progress on the LEP reflects a well integrated program at
LLNL that has maximized utilization of heavily subscribed resources.
NNSA continues to encourage incorporation of LLNL capability in support
of LANL hydrotest requirements to the maximum extent possible. (NA-11
Crandall)

  ---------------------------------------------------------- -----------------
  **Performance Measure 6.3**                                **Outstanding**

  **Develop and implement, with NNSA and other appropriate   
  DOE programs, plans to support optimal use of scientific,  
  research and test facilities and capabilities (e.g., NIF,  
  DARHT, Terascale Computing Facilities, LANSCE) at both     
  Laboratories**                                             
  ---------------------------------------------------------- -----------------

The LLNL self-assessment is accurate but failed to mention that code
scaling studies have been slow to materialize. The need for and ability
to scale codes to the size of the Purple machine depend on a more
aggressive code scaling program. (NA-114)

**NIF**

The Laboratory did an outstanding job of developing and implementing
plans for optimal use of the National Ignition Facility (NIF).

[NIF Demonstration Program:]{.underline}

In FY 2003 the NIF Demonstration Program (NDP) was notable for the
achievement of world- record breaking laser performance using the first
four laser beams available under NIF Early Light. The NDP supported NIF
commissioning and laser performance shots throughout FY 2003. NIF
conducted laser shots into the Roving Mirror Diagnostics Enclosure
calorimeters and the Precision Diagnostics System in Switchyard #2. In
addition, a number of laser shots were fired onto targets at the target
chamber center to conduct energy, timing, and pointing experiments. NIF
delivered over 170 system shots during FY 2003 with excellent results.

NDP quarterly reports were submitted to NNSA on schedule and the content
was consistent with NNSA expectations. All 28 DOE/NNSA performance
milestones scheduled for completion in FY 2003 were completed on or
ahead of schedule. The third quarter FY 2003 actual cost/Budget Outlay
(BO) was 2% under the plan.

[NIF Director Programs:]{.underline}

The National NIF Diagnostics Program (NNDP) activities for FY 2003
focused on installation of the first experimental diagnostics for NIF,
including three Diagnostic Instrument Manipulators (DIMs), the Back-up
Streaked X-ray Detector (SXD-B), and the Static X-ray Imager (SXI). In
addition, the Precision Diagnostics System (PDS), and Full Aperture
Backscatter System (FABS) were installed and commissioned. These
facility diagnostics were utilized with a high degree of reliability
during the NIF Early Light commissioning shots. NNDP quarterly reviews
were held as planned on January 30, 2003 (1^st^ Quarter FY 2003), April
16, 2003 (2^nd^ Quarter FY 2003), and July 22, 2003 (3^rd^ Quarter FY
2003). Good progress continues to be made on development of other NIF
diagnostics and diagnostic support systems, including collaboration with
the NNDP partners (LANL, NRL, etc.).

The National NIF Cryogenic Target Systems Program (NCTS) activities were
scaled-back during FY 2003 because of limited funding. LLNL submitted to
NNSA its first NCTS semi-annual report on September 8, 2003 summarizing
progress for the first half of FY 2003. In the 4^th^ quarter of FY 2003,
a NCTS Statement of Mission Need (for Critical Decision 0) was drafted
by the LLNL, and is currently being reviewed by the NNSA. In FY 2003,
the ICF Campaign 10 Semiannual Meeting was held on July 8-10, 2003 and
LLNL reported its progress on Indirect Drive Ignition.

Issues and Concerns:

[NIF Director Programs:]{.underline}

\(1\) The NIF Director is working to build a national program that will
support the requirements of NIF stakeholders (e.g. weapons physics,
ignition, direct drive, etc.) and optimally utilize the resources of the
Laboratories. Continuing effort is needed to fully integrate a national
program that effectively addresses the needs of the NNSA and its various
stakeholders within the resources available under the FYNSP.

\(2\) The limited FY 2003 budget for NCTS has resulted in further delays
to the completion of the NCTS indirect drive capability.

**Operations**

  ---------------------------------------------------------- ------------------
  **Performance Objective 7**                                **Satisfactory**

  **Maintain a secure, safe, environmentally sound,          
  effective and efficient operations and infrastructure      
  basis in support of mission objectives**                   
  ---------------------------------------------------------- ------------------

  ---------------------------------------------------------- -----------------
  **Performance Measure 7.1**                                

  **a. Meet facility short and long term needs to support    **Outstanding**
  mission requirements: critical facilities, including       
  nuclear facilities, will meet operational needs for        
  programmatic work requirements by minimizing unplanned     
  system outages and downtime**                              

  **b. Meet facility short and long term needs to support    **Outstanding**
  mission requirements: achieve the objectives in the        
  approved FY 2003 Ten-Year Comprehensive Site Plan          
  (TYCSP)**                                                  
  ---------------------------------------------------------- -----------------

Measure 7.1a

The Laboratory did an outstanding job in achieving the FY 2003 goals and
far exceeded the performance indicator; "Availability of all RTBF
critical facilities at 90% of the estimated availability. Facility
availability is defined for each facility as 100% of the planned
capacity, minus the percent of the total possible days not capable of
doing programmatic work due to some unforeseen reason."

The aggregate availability of the RTBF facilities was 99.5% (October
2002 through August 2003). There were no negative impacts on meeting
programmatic milestones. A successful reprogramming addressed previously
un-programmed mandates and unexpected transfer of costs in FY 2003. The
reprogramming, although delayed, was completed in time to avoid the
shutdown of RTBF facilities. Performance requirements were met.

RTBF funds warm stand-by of a sub-set of mission essential facilities
operated by DNT program which directly supports the Stockpile
Stewardship Program. In an effort to reduce the space charges (an
institutional tax based on square footage levied to pay for the real
property maintenance of facilities), RTBF consolidated programmatic
activities from 19 building complexes into 17 building complexes during
FY 2003. Facility availability performance data accounts for this
reduction in the number of RTBF facilities.

Issues and Concerns:

> The continual challenges in maintaining RTBF facilities will be timely
> reprogramming of needed funds, which is a continual possibility in FY
> 2004, and the refurbishment or replacement of aging programmatic
> equipment. While the FIRP is currently addressing the facility real
> property shortfalls, there is no equivalent program for programmatic
> equipment. Currently, the program is continually assessing the
> tradeoffs between programmatic equipment and programmatic
> deliverables.

Measure 7.1b

The Laboratory did an outstanding job in achieving the goals in the FY
2003 TYCSP and far exceeded the performance indicators below.

**Performance Indicators:**

-   Reduction of deferred maintenance backlog to meet industry standards
    by FY09. Current objective is no

> net growth in maintenance backlog

-   Reduction of 90% of the planned gross square footage (24,400 gross
    sq. ft.)

> The Laboratory has already met the NNSA FY 2005 goal of stabilizing
> the deferred maintenance backlog through the aggressive reinvestment
> program that began in the 1990's. LLNL had no deferred maintenance net
> growth in FY 2003 which is two years ahead of the NNSA goal. With the
> projected out year budgets, the LLNL maintenance prediction tool is
> showing that LLNL will achieve the reduction of deferred maintenance
> backlog to meet industry standards by FY09.

Also, LLNL maintenance management system and prediction tool is being
used as a model for NNSA. A workshop on the system and model was
presented in November to all NNSA sites. LLNL has sent a team to LANL to
advise in their establishment of an LLNL comparable maintenance
management system and has provided similar aid to Pantex. LLNL and LANL
have agreed to have continuing reciprocal assessment of our maintenance
management systems.

> Elimination of excess facilities and cleanup of legacy equipment
> and/or material reduces operating costs. About 400,000 sq. ft. of
> space has been removed to date including nearly 130,000 sq. ft. in FY
> 2002. During FY 2003 almost 26,000 sq. ft. have been completely
> removed with an additional 45,000 sq. ft. of demolition projects
> underway that will complete in FY 2004. LLNL achieved the 26,000 sq.
> ft even though the execution of FIRP demolition projects was delayed
> due to the Congressional requirement to Nationally compete all FIRP FY
> 2003 demolition work.

Issues and Concerns:

> LLNL's maintenance backlog reduction modeling program predictions of
> achieving NNSA's FY 2009 goals are based on the FY 2003 FYNSP target
> funding levels. Continued funding of the FIRP program is essential to
> backlog reduction.

  ---------------------------------------------------------- ------------------
  **Performance Measure 7.2**                                

  **Achieve continual improvement in ISM:**                  

  **Develop and implement simplified facility safety basis   **Excellent**
  and related operational requirements for non-nuclear       
  facilities based on benchmarking of best practices**       

  **Assure consistent application of ISM principles across   **Satisfactory**
  all organization levels and across all Laboratory          
  facilities**                                               
  ---------------------------------------------------------- ------------------

In the area of developing and implementing simplified facility safety
basis and related operational requirements for non-nuclear facilities,
the progress being made on developing and implementing this important
program has been **excellent** and continues to exceed LSOD's
expectation.

The WSS for Safety Basis Requirements for Nonnuclear Facilities at LLNL,
Revision 1, UCRL-ID-150214 was approved in February 2003 and
subsequently entered into the LLNL contract. The Pilot for the WSS was
completed with positive feedback being received and incorporated into
the development of the ES&H Manual. A draft of the ES&H Manual, Document
3.1, Safety Analysis Program, was provided to LSO to begin reviewing as
scheduled. Training is being developed and the Q value list in support
of the analysis is almost completed. A high level implementation plan
was completed by LLNL with input from LSO and will continue to be
refined as more information is known. Additionally, the LLNL program was
presented to the recent Safety Analysis EFCOG conference and received
positive feedback from the complex.

The Laboratory performance was **Satisfactory** in assuring consistent
application of ISM principles across all organization levels and across
all Laboratory facilities. LLNL management demonstrated commitment in
maintaining the implementation of ISMS across the institution, however
there are still improvements to be accomplished in some area of ISMS.

-   **Activities Made Since the 2002 ES&H Safety Inspection (SEMI)**

> The SEMI identified two ISMS areas needing improvements: 1)
> Identification of hazards and implementation of controls; and 2)
> Feedback and continuous improvement.

-   *[Identification of Hazards and Implementation of
    Controls:]{.underline}* Consistent with the LSO operational
    awareness activities results, the SEMI found that weakness in this
    area, specifically the implementation of the LLNL Integrated Work
    Sheet (IWS) process. The weaknesses range from inconsistent format,
    insufficient detail, to the generic IWS use by some key LLNL
    organizations. LSO validation of the generic IWS (corrective actions
    of LSO operational awareness findings) found improvements in the
    quality of these IWSs, however, their implementation in which
    additional facility specific hazards are identified prior to
    authorizing work, will continue to be a challenge. There was
    occurrence during the performance rating period in which an employee
    was injured while performing work authorized by a generic IWS but
    facility specific hazards were not identified (CO over exposure).
    There was also occurrence when work was performed without the
    authorization of an IWS (chemical explosion at B-322).

> The LLNL Assurance Review Office (ARO) Assessment of ISMS
> Implementation of the IWS Process (August, 2003) found significant
> improvement across the institution demonstrated through the
> development of the electronic IWS (e-IWS Version 2), and the
> commitments from the line programs in implementing this system. The
> challenge for the line program is to ensure that elements of the IWS
> (such as implement of controls, training, etc.) are implemented, and
> that they are effective.

-   *[Feedback and Continuous Improvement:]{.underline}* Although some
    progress has been made in this area, LLNL is still facing many
    challenges in the institutional improvement of this area:

```{=html}
<!-- -->
```
-   Policy was established (ES&H Manual, Document 4.2) to address cross
    cutting and institutional issues, however implementation has not
    been demonstrated. For example, opportunity for improvements in line
    management accountability identified since last year's Institutional
    ISMS Roll-up would have been an appropriate issue to track, but this
    was not accomplished.

-   ARO Deficiency Tracking System Annual Report (March, 2003)
    identified a large number of long term open items. Similar to
    previous year's roll-up, a deficiency continued to be found in that
    several DefTrack items did not have the Estimated and Actual
    Completion Date which prevented meaningful data analysis. The LLNL
    POCM Self-Assessment deferred to address this issue until the new
    DefTrack System is implemented while NNSA/LSO believes that it can
    and should be addressed now under the current DefTrack System.

-   The new DefTrack System, when implemented, appears to improve the
    ability for LLNL organizations to track and trend ES&H
    self-assessment data. The challenge continues to be the
    implementation of this system. Policies regarding the implementation
    of this system (actions which continued to be delayed until next
    performance period) should ensure that this system must be
    implemented at all level of organizations.

```{=html}
<!-- -->
```
-   **Results of NNSA/Livermore Site Office (LSO) Oversight Activities**

> LSO staff and management conducted operational awareness (OA)
> activities during the performance period as part of our oversight.
> Major LSO OA activities consisted of facility walkthroughs,
> observations of work, document reviews, operational readiness reviews,
> program assessments, investigations, etc. Results of LSO OA activities
> were documented in the LSO Functional Information on Safety, Health,
> and Environment (FISHE) System. During the performance period,
> approximately 30% of LSO OA activities resulted in observation of
> deficiencies, most of which were classified as minor non-compliance
> deficiencies. These results were routinely shared with LLNL's facility
> personnel. ISMS binning of the LSO OA results found that the
> deficiencies found were in the identification of hazard and
> implementation of controls; and operations authorization (Guiding
> Principles 5, 6, and 7). This is consistent with the ARO's CY2002 ISMS
> roll-up.

-   **Results of LLNL CY 2002 Report and Annual Independent ISMS
    Assessment**

> The ARO reports identified a weakness in the ability to verify that
> job specific required trainings were completed as required. LSO
> verification/validation of corrective actions confirmed this weakness.
> This area should be improved as the Directorates upgrade their IWSs
> using the Institution e-IWS Version 2. LSO OA activities during the
> next performance period will focus on the progress in this area.
>
> The ARO report also found that there continued to be inconsistencies
> in the quality of the Directorates' Self-Assessment Reports, which as
> stand alone documents, they have minimal value to the institutional
> ISMS assessment. LSO review of the reports confirmed this finding. LSO
> expects this issue to be addressed by LLNL management.

Issues and Concerns:

-   Although the LLNL had demonstrated improvement in the "paper"
    implementation of the IWS process via the development and
    implementation of the Institution e-IWS Version 2, there is a
    continued challenge as the Directorates revise the IWSs using this
    new system. The other challenge is for the Directorates to ensure
    that elements in the IWS are implemented prior to work being
    performed.

-   Although there are a number of formidable steps in the development
    of the new DefTrack System, LLNL should continue to enforce and hold
    line management accountable for implementing the current DefTrack
    System.

  ---------------------------------------------------------- ------------------
  **Performance Measure 7.3**                                **Satisfactory**

  **Comply with 10 CFR 830 subpart B for the operations of   
  the Laboratories' category 2 and 3 nuclear facilities by   
  completing the required Documented Safety Analysis and     
  Technical Safety Requirements according to the Approved    
  Submission Plan**                                          
  ---------------------------------------------------------- ------------------

Overall the Laboratory performed satisfactorily under performance
measure 7.3 in context of the nuclear safety program. There were
contrasting performance levels associated with this performance measure
primarily due to preparation of several large documents under 10CFR830
Subpart B that involved a large amount of the Laboratory's nuclear
safety analysis resources. A lack of planning to focus on the larger
documents early in the process, resulted in limited attention and
reduced quality of submittals for the nuclear safety program. The first
area, preparation of the required DSAs and TSRs, in all cases was
completed on time. While there were some LSO/DOE conditions of approval,
the documents met most of the LSO quality standards for compliance with
10CFR830 Subpart B and were viewed as adequate. However, for Radioactive
and Hazardous Waste Management facilities and B-334, safety analysis
submittals required extensive discussions between LLNL and DOE. For RHWM
facilities, agreement for comment resolution may have been reached in
these discussions, however the revised document did not always reflect
these changes and resulted in schedule delays for additional resolution.
The development of the new hazards analysis for B-332 indicated
substantial improvement over the existing hazards analysis.

The second area, safety basis amendments were highly variable in
quality. In most cases, the Laboratory omitted some hazards, the
identification of hazardous events or identification of controls. This
resulted in very few amendments being approved without modification to
the submittal or LSO conditions of approval. There were major issues
with the B-231V exhaust modification. The B-332 safety basis amendment
associated with the safety-significant PC2/PC3 change was well done.

Third, the efforts on implementation of the LLNL USQ procedure have
predominantly been poor. Technical bases in USQDs sometimes do not lead
an independent reviewer to the same conclusion as the preparer.
Significant USQ implementation issues were identified in B-231V during
Fall of CY 2002. The B-231V facility was slow in response and continued
to have difficulty in understanding fundamental nuclear safety
principles. While the Laboratory did a good job in identifying USQ
implementation issues through the ARO assessment, the Laboratory was not
aggressive to resolve those assessment issues. In many cases the LLNL
corrective actions did not resolve the issue or look systematically at
the corrective actions that would resolve the issue.

Of greater concern is the extended time that DOE Order 5480.21 (USQ
Process) has been in the UC contract (early 1990s) as well as the time
associated since the implementation of 10CFR830, Subpart B for the USQ
process (April 10, 2001). LLNL has been very slow in implementation of
the USQ process relative to the rest of the DOE Complex. As a result,
changes are not always evaluated or authorized appropriately. In some
cases DOE has not been aware or authorized some incremental risks. Those
risks have been accepted by LLNL instead of DOE in greater than Hazard
Category 3 nuclear facilities. More specific discussion follows.

Issues and Concerns:

In the past fiscal year there were some areas that the Laboratory
performed adequately. In particular, the Laboratory completed four
DSA/TSRs submittals under 10CFR830. The Laboratory did not experience
any nuclear safety incidents resulting in off-site consequences or
severe worker injury. Recognizing that this was the final transition
year for 10CFR830 Subpart B, and the intense effort that went into
preparation of the compliant DSA and TSR submittals, LSO looked at the
overall performance of the nuclear safety program. Implementation of
safety analysis documents overall was adequate but lacked particularly
in B-231V facility. Overall implementation of the USQ process and
development of compliant safety basis amendments continued to lag behind
the rest of the DOE Complex. There continues to be nuclear safety areas
that LLNL has not effectively implemented that are well past their
associated Rule implementation dates. Those areas are:

1)  safety basis amendments quality and planning---this includes
    comprehensively addressing all hazards, identification of hazardous
    events and identification of controls

2)  slow implementation of the compliant USQ procedure including lack of
    technical bases for USQ determinations

3)  understanding of "change" associated with the USQ process

4)  lack of comprehensive development of corrective actions plans that
    address issues in nuclear safety.

5)  lack of self validation and oversight of nuclear safety at
    LLNL---normally this oversight is prompted by LSO and not the
    Laboratory.

  ---------------------------------------------------------- ------------------
  **Performance Measure 7.4**                                **Satisfactory**

  **Complete the NNSA-approved action plans and UC-approved  
  project plans for implementing Integrated Safeguards and   
  Security Management and after that, achieve continuous     
  improvements by providing consistency throughout the       
  Laboratory**                                               
  ---------------------------------------------------------- ------------------

LSO verified that LLNL completed the initiatives planned by the LLNL
Implementation Guidelines for FY 2003 Appendix F. The initiatives closed
three activities initially planned under Appendix O in FY 2002, and
produced an evaluation report of ISSM by LLNL. The evaluation identified
many LLNL ISSM issues, most of which pertained to employees' lack of
awareness of responsibilities or security requirements. The evaluation
report noted that these issues did not involve significant security
risks.

The LLNL FY 2003 Appendix F Annual Self-Assessment refers to an issue of
"two security key incidents and an unsecured property protection area
gate" in which there was no "evidence or indication of an actual threat
to classified information". The security key incidents led to reviews by
LSO, NNSA HQ, and LLNL that found systemic deficiencies in LLNL
institutional security key control and inventory, and security incident
handling. Deficiencies in security key control and inventory increase
the risk of unauthorized access to classified information. Deficiencies
in security incident handling resulted in a below-listed Finding based
on DOE Policy 470.1, Integrated Safeguards and Security Management.

\- Measures and controls related to LLNL incident reporting are not
sufficient to ensure that risks related to incidents are reported in a
timely manner, mitigated as warranted, and accepted by line management.

Other conditions relevant to the LSO evaluation of Appendix F include
the following two Findings issued by LSO to LLNL that pertain to ISSM
implementation deficiencies.

-   LLNL has not applied the principles and components of Integrated
    Safeguards and Security Management to its COMSEC activities. 
    Specifically, lines of authority and responsibility for LLNL COMSEC
    are ambiguous and undocumented; COMSEC activities are not fully
    integrated with other related security activities; and COMSEC
    performance is not institutionally monitored and assessed. 

\- The guiding principles and core functions of Integrated Safeguards
and Security Management are not sufficiently applied to systematically
integrate safeguards and security management and work practices to
prevent recurring security incidents in Superblock operations.

\- LLNL Classified Matter Protection and Control practices do not ensure
that all classified removable electronic media are subjected to
accountability controls as required. LLNL accountability inventory
processes do not address all material that should be accounted for and
inventoried.

Issues and Concerns:

LSO expressed concerns about the two above-listed ISSM deficiencies
several months before the Findings were issued; however, during these
months, LLNL corrective actions were not taken or completed as planned.
The deficiencies were not considered by the LLNL FY 2003 ISSM continuous
improvement initiatives or the LLNL FY 2003 Appendix F Annual
Self-Assessment. It is also a concern that LLNL self-assessment
activities did not reveal significant discrepancies in the fundamental
requirements of security key control and inventory, and accountability
of classified removable electronic media before the occurrence of
security incidents and the issuance of Findings.

Several security incidents involving the LLNL protective force (e.g.,
unsecured gates, entry control errors) were reported during the last few
months of the Appendix F rating period. This demonstrates increased
awareness of security incident reporting. Along with deficiencies in
LLNL protective force staffing and training, this condition also
demonstrates a need to apply further ISSM continuous improvement
initiatives to LLNL Safeguards and Security and Protective Force
activities.

  ---------------------------------------------------------- ---------------
  **Performance Measure 7.5**                                **Excellent**

  **Develop with NNSA a long-term plan to reduce inventories 
  of surplus and excess SNM and onsite waste**               
  ---------------------------------------------------------- ---------------

LLNL performance was excellent because of its participation in the
completion of the Inactive Actinide Working Group activities. LLNL has
identified, in detail, what Pu and HEU items can be transferred to SRS.
LLNL submitted two technology projects to "clean" surplus materials
which will enable timely disposition of the materials. LLNL is the lead
for part 2 of the overall strategy to correct deficiencies in the
management of inactive actinides which is the development of a
characterization methodology to ensure safe storage and support
disposition decisions. LLNL has been working with the contracted mobile
vendor to process LLNL's TRU legacy waste, so waste can begin moving
from LLNL to WIPP in FY 2004.

Issues and Concerns: LLNL did not verify it had in its evidence file all
supporting documentation, such as project plans, characterization plan,
etc.

  ---------------------------------------------------------- ---------------
  **Performance Measure 7.6**                                **Excellent**

  **Maintain an Environmental Management Program consistent  
  with DOE negotiated regulatory requirements and funding    
  levels (LLNL)**                                            
  ---------------------------------------------------------- ---------------

**Waste Management:**

LLNL accomplished all their goals for treatment and disposal of mixed
low-level, low-level and hazardous wastes for FY 2003. Regulatory
commitments for treatment and disposal for mixed low-level wastes were
also met. Other specific accomplishments, listed below contribute to the
rating of Excellent.

● DWTF Readiness was on schedule. DWTF started operations based on EM-1
conditional approval.

● LLNL met EM headquarter goals for Legacy Waste Inventory Reduction for
FY 2003. The amount of Legacy Waste eliminated from the Legacy Waste
inventory, surpassed the EM HQ goal of 375 cubic meters.

● LLNL met EM headquarter goals for Newly Generated Waste disposal for
FY 2003.

● LSO Integrated Project Baseline was recommended for approval by EM-6.

**Environmental Restoration:**

LLNL performed in an excellent manner in Fiscal Year 2003 in its efforts
to complete enforceable agreement (EA) milestones on or ahead of
schedule to support the buildout of the required remediation systems at
the Livermore Site and Site 300. Buildout of the remediation systems for
the environmental restoration projects at both sites is essential to
achieve EM Completion. LLNL completed all (3) EA milestones for the
Livermore Site and all (6) EA milestones for Site 300. Completion of all
milestones was significant as a majority of LLNL project management's
time was concerned with updating lifecycle baselines per newly developed
EM guidelines and requirements for project control systems.

Environmental restoration operations conducted by LLNL met their
regulatory requirements and LLNL completed all negotiated enforceable
agreement milestones on time and within funding levels to support
buildout of the required remediation network system. LLNL restoration
facilities and operations were managed to protect human health and the
environment while maximizing the efficient use of EM funds. All
treatment facilities operated normally and in compliance with
Substantive Requirements for Wastewater Discharge.

  ---------------------------------------------------------- ------------------
  **Performance Measure 7.7**                                **Satisfactory**

  **Implement an Emergency Management Program within NNSA    
  approved schedules (LLNL)**                                
  ---------------------------------------------------------- ------------------

The Laboratory did a satisfactory job implementing an Emergency
Management Program within an NNSA approved schedule. Improvements
included:

-   Submitted a Project Management Plan (PMP) to address major
    milestones to achieve compliance with Order 151.1 through FY 2004.
    The PMP was submitted on March 14, 2003, resubmitted on July 21,
    2003 and approved by LSO on July 31, 2003.

-   Submitted the required annual updates to the LLNL Emergency Plan and
    the Emergency Plan Implementing Procedures (EPIPs). The LLNL
    Emergency Plan was approved by LSO on March 17, 2003. The EPIPs have
    been through comment resolution with LSO and are considered
    acceptable.

-   Submitted 6 Emergency Preparedness Hazards Assessments (EPHAs) for
    LSO approval. See chart below for approval status.

  ---------------- ------------------ ------------------ -----------------
  EPHA             LLNL Submittal     LSO RCR to LLNL    LSO Approval

  B153 EPHA        3/28/03                               4/11/03

  B695 EPHA        6/26/03            7/3/03             7/24/03

  TWLP EPHA        9/11/03                               9/24/03

  B166 EPHA        9/30/03                               10/6/03

  B197 EPHA        9/30/03            10/10/03\*         

  A514 EPHA        9/30/03            10/14/03\*         
  ---------------- ------------------ ------------------ -----------------

> \*Minor comments only

-   Conducted the required annual exercise, which included validation of
    the SEMI CAP Emergency Management findings. The annual exercise was
    conducted on May 21, 2003. Overall the exercise adequately
    demonstrated the LLNL's ability to mitigate and manage a potential
    emergency situation. A corrective action plan was submitted to LSO
    on August 1, 2003 to address exercise weaknesses and deficiencies.

-   LSO conducted an independent, limited scope evaluation of LLNL's
    annual exercise. Overall, LLNL's and LSO's evaluation were
    consistent. Two areas of superior performance, 5 opportunities for
    improvement, 3 weaknesses and 1 deficiency were identified. A
    corrective action plan was submitted to LSO on August 1, 2003 to
    address weaknesses and the deficiency.

  ---------------------------------------------------------- -----------------
  **Performance Measure 7.8**                                **Outstanding**

  **Ensure effective controls in business systems by         
  assessing existing controls and, where needed,             
  strengthening controls to ensure effective stewardship of  
  public assets**                                            
  ---------------------------------------------------------- -----------------

Lawrence Livermore National Laboratory performance in this measure earns
an "outstanding" rating for FY 2003. The laboratory has, and practices,
cost effective business and financial systems and controls. Appropriate
management control systems are in place and functioning as intended.
LLNL has consistently been highly responsive to new and changing
DOE/NNSA procurement, property, and financial management requirements.
Cost accounting practices are sound, effective and support federal
stewardship objectives.

Highlights in Financial Management for FY 2003 include LLNL completing
all requested budget schedules in a timely, accurate, and complete
manner.  An outstanding rating on accounting practices is warranted
because LLNL matched or improved its performance in all except one
(discounts) of the accounting processes measured. In addition LLNL
pro-actively addressed new DOE initiatives and emerging issues and was a
leader in facilitating communications within the NNSA Office of Field
Financial Management and other organizations.

Highlights in Property Management for FY 2003 include completing the
wall-to-wall inventory of Equipment and Attractive items which resulted
in an accountability rate of 99.8%. On-going operational awareness
indicated that the LLNL Property Management system continues to perform
commendably in integrated areas such as: property tagged in receiving,
property field-tagged within 5-days, property identified via electronic
purchasing system, and sub-contractor-held property. LLNL Property was
the subject of several audits by third party entities such as the OIG
during the year. No systemic weaknesses were identified.

Highlights in Procurement for FY 2003 include: the contractor having a
comprehensive self assessment and review process that ensured compliance
with internal and external policies and procedures. Procurement and
Materiel (P&M) measured its effectiveness in the eight Objective Matrix
areas by conducting risk based self assessments of its purchase orders,
subcontract and agreement transactions, and procurement transactions of
its Laboratory's Technical Release Representatives (TRRs). LSO validated
the Contractor's self-assessment of its purchase orders and subcontracts
for FY 2003, which included a PARIS system review. P&M's self-assessment
addressed five contractual areas: Source Selection, Pricing Methodology,
Quality of Contracts and Contract Administration. They identified three
systemic findings, with the average score for these files being 91.9%,
and corrective action was appropriately implemented. UC commissioned
Ernst & Young (E&Y) independent assessment of internal controls and the
internal Audits and Oversight (A&O) review recommendations were
validated by P&M and LSO. No systemic issued were identified and the
recommended enhancements are in the process of being implemented over
the next two fiscal years.

  ---------------------------------------------------------- ---------------
  **Performance Objective 8**                                **Excellent**

  **Utilize UC strengths to recruit, retain and develop the  
  workforce basis**                                          
  ---------------------------------------------------------- ---------------

  ---------------------------------------------------------- ---------------
  **Performance Measure 8.1**                                **Excellent**

  **Provide skills necessary to enhance the science base by  
  implementing integrated recruiting and retention           
  strategies to meet the Laboratories' long-range skills     
  requirements**                                             
  ---------------------------------------------------------- ---------------

> LLNL performed at the "**Excellen**t" level under this measure in
> FY2003. The Laboratory addressed this measure through the
> implementation of several strategies -- the annual workforce review to
> identify skill gaps and workforce issues within each Directorate and
> lab-wide, completion of initiatives responsive to Employee Survey
> findings to promote retention, and an institutional recruitment effort
> to incorporate the UC image and address the pipeline recruitment needs
> identified through the work force reviews. In addition, responsive to
> feedback received at mid-year, LLNL addressed through its
> self-assessment the status of its efforts to ensure core and critical
> skills requirements were identified and monitored, to provide
> continuity from Appendix O.
>
> The implementation of an annual workforce review in FY2003 was the
> primary vehicle by which LLNL assessed the status of its workforce
> needs, in terms of recruitment/retention requirements as well as
> employee development, diversity, and quality of leadership. The
> Laboratory Director and his senior management team conducted the
> reviews of each Directorate, and then assessed the issues that
> required an institutional strategy to address. The status of the
> following topics was required for the briefings to the Director by
> each Assistant Director:

-   Workforce demographics

-   Workforce capability (core skills, including strengths and skill
    gaps)

-   Salary management

-   Managerial and technical leadership development and succession
    planning

-   Security

-   Work environment

-   Institutional strategic initiatives

-   Diversity

-   Overall employee development

-   Safety

> The identification of core and critical skills, and determination of
> the current and future skill gap was central to the Lab Director's
> discussion with each Directorate. Because of the fluid nature of
> LLNL's matrix assignments, through the reviews the appropriate ratio
> of critical skill positions relative to other positions was identified
> within each Directorate, in order to establish a means of ensuring the
> critical skills needs were being met. Subsequent to the workforce
> reviews, Human Resources staff met with each Directorate to discuss
> the core/critical as well as other staffing and skills mix issues/gaps
> that were identified, demographics and diversity needs, success of FY
> 2002's recruitment/advertising efforts, and recruitment best
> practices, strategies and sources to attract/retain a diverse
> workforce. The results from these meetings formed the basis for
> developing the recruitment plans and objectives for the year. The Lab
> maintained the status of the core/critical skill and pipeline
> populations in a database, which allowed them to assess the status by
> series, discipline, directorate, age, service, and the open
> requisitions generated from HR's discussions with the Directorates.
> The ratio of critical skills to the general population (200, 300, 900
> series employees) was applied to the open requisitions to quantify the
> Laboratory's critical skill gap and measure the progress toward
> filling the gap.
>
> The integrated, institutional recruitment strategy was developed to
> maximize the three UC Laboratories' relationships and common
> recruitment objectives, and their affiliation with the University of
> California. During FY2003, LLNL developed Campus Accelerated
> Recruiting in Engineering (CARE), for implementation in FY2004. CARE
> was based on an initiative LANL had previously implemented, in which
> candidates for Engineering positions went through two days of on-site
> interviews as a group to meet and interview with hiring managers in a
> mini job fair setting. Within two days of the event, managers made
> offers or non-selection letters were sent. Several benefits of this
> interview process were realized, including better facilitation
> selection decisions and closure with candidates, broader exposure for
> candidates, and savings in staff time and travel costs. Another
> initiative based on LANL's program was the expansion of technical
> recruiter training, which provided three half-day workshops to
> directorate technical recruiters to address the knowledge, skills and
> resources they identify that are lacking. Other activities, involving
> all three UC Laboratories, included development of a Tri-Lab
> recruitment image display, participation in joint recruitment
> activities at UC campuses, and participation in Tri-Lab Day at UC
> Berkeley, with others scheduled for the Spring at UC Berkeley and UC
> Santa Barbara.
>
> Also under the Integrated Institutional Recruitment strategy, LLNL
> established "UC Pipeline Recruitment" efforts to attract new talent
> and create a pipeline for critical skill positions through initiatives
> and programs involving UC campuses. (ADD DATA on HIRES) The following
> table provides a description of and LLNL's achievements under these
> programs:

+---------------+------------------------------------------------------+
| **Program**   | **Description**                                      |
+---------------+------------------------------------------------------+
| UC Student    | • Fellowships are awarded to students to carry out   |
| Employee      | original research at LLNL in pursuit of their Ph.D.  |
| Graduate      | in subjects related to the goals and missions of the |
| Fellowship    | Laboratory. The FY 2003 SEGRF roster consists of 70  |
| (SEGRF)       | students from six of the nine UC campuses. This      |
|               | fiscal year a detailed analysis of the program,      |
|               | showed that LLNL has continually retained 33% of     |
|               | these students as employees following their          |
|               | fellowship. Initiated in 1964, this program          |
|               | originally was only offered to UC Davis graduate     |
|               | students. It is now offered to UC students system    |
|               | wide.                                                |
+---------------+------------------------------------------------------+
| Distinguished | • Program provides research opportunities for        |
| Lawrence Post | outstanding researchers to pursue cutting-edge       |
| Doctoral      | science. In FY 2003, there were more than 400        |
| Fellowship    | applicants, two candidates have accepted and six     |
| Program       | continue their fellowships. Since the program        |
|               | inception six years ago, 17 fellows were selected,   |
|               | eight have become career employees. The program also |
|               | provides a finalist pool for recruiting, which has   |
|               | resulted in 50 hires.                                |
+---------------+------------------------------------------------------+
| Critical      | • Administered through LLNL\'s University Relations  |
| Skills        | Program (URP) Internship program provides            |
| Internship    | opportunities for under graduate and graduate        |
| Program       | students in Chemistry, Materials Science, Computer   |
|               | Science, Engineering, and Physics - \"critical       |
|               | skills\". In FY 2003, the program included ten       |
|               | individual internship programs with a student        |
|               | population of 84.                                    |
|               |                                                      |
|               | • LLNL\'s University Relations Program (URP)         |
|               | administers summer programs. In addition to summer   |
|               | research experience, there were 123 events arranged  |
|               | for summer students. Events ranged from lectures,    |
|               | tours, panel discussions, UC campus visits and GRE   |
|               | preparation sessions, culminating in a Lab-wide      |
|               | summer student symposium. One third of the students  |
|               | are in programs identified by NNSA as being          |
|               | critical.                                            |
+---------------+------------------------------------------------------+
| Sabbatical    | • Program provides research opportunities for        |
| Program       | internationally known faculty and for highly         |
|               | qualified graduate students. In FY 2003, there were  |
|               | a total of eight visiting faculty and eleven         |
|               | students and postdocs. This program is supported     |
|               | using UC award fee funds.                            |
+---------------+------------------------------------------------------+

> Implementation of the Employee Survey Action recommendations has been
> the focus of LLNL's retention strategy since 2000, when the survey was
> taken. With the exception of two construction projects (sport court
> and central cafeteria), all of the other 26 recommendations were
> implemented by the end of FY 2003. Those most relevant to this measure
> are implementation of the Uniform Performance Management Program
> (IPPP), which provides a consistent methodology for ranking across the
> lab and more effective salary management, Institutional Guidelines for
> Career Development, and the piloting of Directorate Career Development
> Programs for the development of technical competencies.

  ---------------------------------------------------------- ---------------
  **Performance Measure 8.2**                                **Excellent**

  **Implement leadership and management development programs 
  aligned with workforce planning and diversity objectives** 
  ---------------------------------------------------------- ---------------

LLNL performed at the **Excellent** level under this measure in FY2003.
LLNL developed a comprehensive leadership and management development
strategy over the past few years that provided for the developmental
needs of all levels of supervisors and managers, as well as potential
leaders. In addition to a set of core courses (Supervision I and II,
Management Institute), the Laboratory's Leadership and Management
Development Program offered over twenty general management courses
in-house, such as "Leadership Challenge", "Mastering Tough Decisions",
"Communicating for Results", and "Seven Habits of Highly Effective
People." It also sponsored three leadership/career development programs
specifically addressing issues unique to high potential minorities,
maintained a cadre of recommended external courses for various levels of
leader, including UCLA's Technical Management program, UC Management
Skills Assessment Program, and several from the Center for Creative
Leadership, and offered a highly popular Leadership Lecture Series,
which is open to all employees. In addition, eight of the thirteen
directorates initiated their own leadership development programs to
develop succession pools and enhance the development of their current
management teams. All courses were connected to developing skills,
knowledge or abilities under one or more of the seven core competencies
the Laboratory has identified for a well-rounded, effective leader.
Those seven core competencies are: Strategic Thinking, Political and
Cultural Acumen, Management, Leadership, Interpersonal Effectiveness
(Emotional Intelligence), Communication, and Self-Management.

It is through the individual Directorate Leadership Programs that LLNL
ensured alignment with workforce planning and diversity objectives. The
work force reviews conducted in 2003 of each Directorate required the
identification of individuals with the potential for becoming managers
and technical leaders, and placed the expectation on Directorate
management to implement the strategies necessary to develop this
"succession pool". While several directorates already had leadership
development programs in place, the targeted focus on succession planning
through the work force reviews significantly expanded the number of
programs and attention provided to this area. Directorates worked with
Human Resources staff to customize a leadership program appropriate to
their needs and participant level, drawing from the core, open
enrollment, and external courses and Leadership Lecture Series topics to
tailor a program that provides assessment and feedback, skills
development, group projects, and individual coaching. Participant groups
generally have 20-25 members, who complete a series of activities
together over a period of six months to a year. Although at present
eight of the thirteen directorates have directorate-specific programs,
the Laboratory has also established an institutional group to provide
coordination across all directorates and networking to share ideas and
lessons-learned, and leverage access to external speakers/consultants.
Human Resources tracks the progress of participants, enabling the
Directorates to report out the results of their succession planning in
the annual work force review. To date, over 330 employees have
participated in these Directorate Leadership Programs, with over 45%
having been promoted or having assumed increased levels of
responsibility since completing the programs. Of the participants, 33.3%
have been females and 16.5% minorities, and of those promoted or who
have received increased responsibilities, 25% have been females and 15%
minorities. These figures reflect relatively proportional representation
of females and minorities to those of the overall Laboratory, at 31% and
19%, respectively.

At the more senior management level, the Management Institute is a very
successful program for "preparing the next generation of Laboratory
Leaders". The Institute is a two and one-half day program provided
annually, with the goal of broadening the participants' "understanding
of current issues facing the Lab, institutional management
responsibilities, and strategies for leading the Laboratory into the
future." Issues include Lab culture, the political process, the future
of Lab programs and direction of science, and the operations of a
mission-based R&D lab. The Laboratory Director, as well as the Deputy
Directors and other Senior Management Group members, form the faculty of
the Institute, whose participants are nominated by their Associate
Directors and selected by the Director's Office. Subsequent to the
Institute, the Director's Office continues to monitor the development of
alumni by providing guest speakers and seminars, and through utilizing
them as a cadre for managing special projects with broad institutional
implications. To date, 108 employees are alumni of the Management
Institute, with 40% having been promoted or assumed increased
responsibility since participating. Of these 108, approximately 27% were
female, and approximately 16% minority, again relatively proportional to
the overall Lab representation. Of those who have been promoted or
assumed increased responsibility, 23% were female, but only 2% (1
individual out of 43) minority.

Overall, LLNL has developed a successful, comprehensive leadership
development program that addresses future needs at every level, from
potential leaders through senior management. Through the workforce
reviews, succession planning receives senior Lab management attention,
ensuring there is accountability for the development of individuals
identified. The identification of core competencies and the variety of
courses available under each allow for tailoring a program to individual
or organizational needs while providing exposure to the full breadth of
leadership/management. Diversity is maintained at levels close to
lab-wide representation for women and minorities, with the exception of
minority alumni of the Management Institute receiving promotions or
increased responsibilities. This is an area LLNL would benefit from
examining further, in terms of potential impact on the diversity of the
Lab's future senior management team. Aside from this, the *structure* of
LLNL's program positions them well to assure NNSA that a competent cadre
of leaders for the Lab's future will be available.

Issues and Concerns:

Diversity is maintained at levels close to lab-wide representation for
women and minorities, with the exception of minority alumni of the
Management Institute receiving promotions or increased responsibilities.
This is an area LLNL would benefit from examining further, in terms of
potential impact on the diversity of the Lab's future senior management
team.

  ---------------------------------------------------------- -----------------
  **Performance Objective 9**                                **Outstanding**

  **Sustain effective Community Initiatives**                
  ---------------------------------------------------------- -----------------

  ---------------------------------------------------------- -----------------
  **Performance Measure 9.1**                                **Outstanding**

  **Leveraging the UC expertise and mission in science       
  education, the laboratories will establish and maintain    
  science education outreach programs with the joint goals   
  of community outreach and substantive contribution to      
  science education**                                        
  ---------------------------------------------------------- -----------------

**Performance Measure 9.1** - Two organizations contribute to the
accomplishments of this performance measure, the Science & Technology
Education Program (STEP) and the Public Affairs Office (PAO).

Highlights in this area include: 1) STEP that leads the Laboratory's
efforts in K-12 science education, in partnership with the University of
California's Edward Teller Education Center (ETEC), held several
workshops for approximately 500 middle school and high school teachers
to help further their knowledge, skills, and abilities in the area of
science and math; 2) STEP's partnership with UC ETEC also helped expand
UC's K-12 science education into local/regional activities to the
Central Valley by forming partnerships with existing Regional Education
Centers in Fresno and Merced; 3) PAO conducted many successful outreach
activities including a School Tour program for local 4^th^ and 5^th^
grade students in the Tri-Valley area; and two series of community
family-oriented science lectures, *Spotlight on Science* and *A Sizzlin'
Summer Science* both of which had excellent participation and positive
feedback.

  ---------------------------------------------------------- -----------------
  **Performance Measure 9.2**                                **Outstanding**

  **The Laboratory will develop local community initiatives  
  to include those programs or responses addressing mutual   
  goals and concerns (LLNL)**                                
  ---------------------------------------------------------- -----------------

**Performance Measure 9.2** -- Responsibility for this measure rests
with the Public Affairs Office. PAO did an outstanding job in developing
a number of new community outreach initiatives to address areas of
potential concern to the community. Highlights include: 1) The Lab
hosted a Community Leader Day for elected and appointed community
officials to better acquaint them with the operations of the Laboratory
and the NNSA's Livermore Site Office. This was a very successful event
with approximately 180 local dignitaries participating. 2) Working in
partnership with the Sandia National Laboratory and the Livermore Site
Office, PAO developed a thorough communication plan for the public
closure of East Avenue. The public closure went very smoothly which was
due to PAO's proactive communications with neighbors, city officials,
businesses and lab employees; and 3) PAO partnered with the County of
Alameda and the cities of Livermore, Pleasanton and Dublin to develop
the *Tri-Valley Community Emergency Response Guide* to provide local
residents with information to prepare for and respond to a
community-wide emergency.

Appendix A

Ratings

**Overall LLNL Rating**

  -------------------------------------------------------- ------------------
  Overall Rating                                           **Excellent**

                                                           

  Mission (Performance Objectives 1-6)                     **Outstanding**

  Operations (Performance Objectives 7-9)                  **Satisfactory**
  -------------------------------------------------------- ------------------

**Rating by Performance Objective**

  --------- --------------------------------------------------- -----------------
  Mission                                                       

  **1.**    Develop and Implement a Common UC Design Laboratory **Excellent**
            Certification Strategy                              

  **2.**    Develop with NNSA and implement long-term balanced, **Outstanding**
            integrated stewardship                              

  **3.**    Develop with NNSA and implement near-term balanced  **Outstanding**
            weapon program                                      

  **4.**    Develop and implement sound non-proliferation /     **Outstanding**
            counter terrorism program basis                     

  **5.**    Enhance and nurture a strong science and technology **Outstanding**
            base in support of NNSA national security           
            objectives                                          

  **6.**    Achieve successful completion of projects and       **Outstanding**
            development of user facilities                      
  --------- --------------------------------------------------- -----------------

  ------------ --------------------------------------------------- ------------------
  Operations                                                       

  **7.**       Maintain a secure, safe, environmentally sound,     **Satisfactory**
               effective and efficient operations and              
               infrastructure basis in support of mission          
               objectives                                          

  **8.**       Utilize UC strengths to recruit, retain and develop **Excellent**
               the workforce basis                                 

  **9.**       Sustain effective Community Initiatives             **Outstanding**
  ------------ --------------------------------------------------- ------------------

Appendix A

Ratings

Ratings by Performance Measure

  -------- ------------------------------------------------------- -------------
  **1.**   Develop and Implement a Common UC Design Laboratory     
           Certification Strategy                                  

  1.1      Develop an integrated, scientifically-based             Outstanding
           quantitative certification methodology that has been    
           externally reviewed for use in future warhead           
           certification and to support the Annual Certification   
           Process                                                 

  1.2      Demonstrate application of a common assessment          Excellent
           methodology using Quantification of Margins and         
           Uncertainty (QMU), in major warhead assessments         

  1.3      Demonstrate progress toward quantifying margins and     Excellent
           reducing uncertainties relevant to primary and          
           secondary performance                                   
  -------- ------------------------------------------------------- -------------

+---+-------------------------------------------------------+----------+
| * | Develop with NNSA and implement long-term balanced,   |          |
| * | integrated stewardship                                |          |
| 2 |                                                       |          |
| . |                                                       |          |
| * |                                                       |          |
| * |                                                       |          |
+---+-------------------------------------------------------+----------+
| 2 | Support the needs of warhead assessment and           | Out      |
| . | certification by coordinated programs of targeted     | standing |
| 1 | small- and large-scale experiments and mining of      |          |
|   | archival UGT data to improve predictive capability    |          |
+---+-------------------------------------------------------+----------+
| 2 | Demonstrate advances in radiography technology and    | Out      |
| . | develop joint options and recommendations for future  | standing |
| 2 | x-ray and proton radiographic capability that support |          |
|   | the quantitative certification methodology            |          |
+---+-------------------------------------------------------+----------+
| 2 | Demonstrate ASC simulation and modeling capabilities  | E        |
| . | that support the ongoing needs of stockpile           | xcellent |
| 3 | assessment and certification                          |          |
+---+-------------------------------------------------------+----------+
| 2 | Improve and apply tools and models for prediction of  | Out      |
| . | systems and/or component lifetimes                    | standing |
| 4 |                                                       |          |
+---+-------------------------------------------------------+----------+
| 2 | Develop and implement a collaborative and             | Out      |
| . | complementary program of experiments at High Energy   | standing |
| 5 | Density (HED) facilities that supports the            |          |
|   | quantitative certification methodology                |          |
+---+-------------------------------------------------------+----------+
| 2 | > Develop an integrated program for plutonium         | E        |
| . | > capabilities of LANL and LLNL to support the        | xcellent |
| 6 | > overall NNSA strategic requirements                 |          |
+---+-------------------------------------------------------+----------+

  -------- ------------------------------------------------------- -------------
  **3.**   Develop with NNSA and implement near-term balanced      
           weapon program                                          

  3.1      As part of the Annual Certification Process, the        Outstanding
           laboratory directors will complete the annual           
           assessments of the continued safety, reliability and    
           performance of all warhead types in the stockpile       
           including whether nuclear testing is required for       
           resolution of any issue; and support DOE as required    
           during interagency and community coordination of the    
           Annual Certification Process                            

  3.2      Provide technical support to production complex         Excellent
           operations, including the Integrated Weapons Activity   
           Plan (IWAP)                                             

  3.3      Deliver on W88 Pit Manufacturing and Certification      Outstanding
           Project major milestones                                

  3.4      Deliver on the major milestones for the Life Extension  Outstanding
           Programs for the W76, the B61-7/11, and the W80- 2/3 in 
           accordance with the Phase 6.X process                   

  3.5      Conduct stockpile surveillance and assessment           Outstanding
           activities, including investigation and subsequent      
           resolution of significant findings on a priority basis, 
           and issues identified in technical assessment reports   

  3.6      Support directive schedule requirements                 Excellent
  -------- ------------------------------------------------------- -------------

  -------- ------------------------------------------------------- -------------
  **4.**   Develop and implement sound non-proliferation / counter 
           terrorism program basis                                 

  4.1      Sustain and expand intelligence and                     Excellent
           counter-intelligence programs analysis and analytical   
           data systems for detecting and thwarting Proliferation  
           and Terrorism                                           

  4.2      Sustain and expand international cooperative programs   Outstanding
           to reduce the threat of nuclear proliferation           

  4.3      Develop and expand complex systems modeling to enhance  Outstanding
           prediction and identification of threats,               
           prioritization and integration of counter-terrorism     
           efforts, and effectiveness of response systems for      
           terror events                                           

  4.4      Develop and transition technologies for large-scale     Outstanding
           deployment for civilian preparedness against terrorist  
           biological, chemical and other attacks                  

  4.5      Develop and demonstrate nuclear detection and           Outstanding
           monitoring technologies; provide technologies and       
           expertise to enhance protection of nuclear materials    
           in, at, and outside of US borders; and maintain the     
           capability to deploy a nuclear emergency response team  
           for protecting US assets from radiological and nuclear  
           threats                                                 

  4.6      Develop global situational awareness with the defense   Outstanding
           and intelligence communities and enable the necessary   
           technical underpinnings to monitor and track            
           facilities, people, and situations worldwide in         
           real-time                                               
  -------- ------------------------------------------------------- -------------

  -------- ------------------------------------------------------- -------------
  **5.**   Enhance and nurture a strong science and technology     
           base in support of NNSA national security objectives    

  5.1      Develop and implement an integrated and balanced        Outstanding
           strategy for investing LDRD, programmatic and           
           institutional resources to ensure the long-term         
           vitality of the Laboratory Science and Technology base  
           to support the NNSA mission and emerging national needs 

  5.2      Nurture and maintain the Laboratories' signature        Outstanding
           capabilities including unique experimental facilities   
           and competencies in support of Laboratory and external  
           users and sponsors                                      

  5.3      Strategically pursue and successfully execute a         Outstanding
           portfolio of non-NNSA sponsored research that builds on 
           unique Laboratory capabilities and enhances the         
           Laboratories' competencies to meet current and future   
           national security needs                                 

  5.4      Foster active participation in the broad scientific     Outstanding
           community and leverage unique Laboratory expertise and  
           capabilities to develop strategic collaborations with   
           other national laboratories, industry and academia      
  -------- ------------------------------------------------------- -------------

  -------- ------------------------------------------------------- -------------
  **6.**   Achieve successful completion of projects and           
           development of user facilities                          

  6.1      Execute significant construction projects as identified Outstanding
           and agreed to between the Site Offices and laboratories 
           within budget, scope and schedule                       

  6.2      Develop with NNSA and implement a National Hydrotest    Outstanding
           Plan that addresses mutual utilization of hydrotest     
           facilities, including containment and materials         
           availability                                            

  6.3      Develop and implement, with NNSA and other appropriate  Outstanding
           DOE programs, plans to support optimal use of           
           scientific, research and test facilities and            
           capabilities (e.g., NIF, DARHT, Terascale Computing     
           Facilities, LANSCE) at both Laboratories                
  -------- ------------------------------------------------------- -------------

+---+-------------------------------------------------------+----------+
| * | Maintain a secure, safe, environmentally sound,       |          |
| * | effective and efficient operations and infrastructure |          |
| 7 | basis in support of mission objectives                |          |
| . |                                                       |          |
| * |                                                       |          |
| * |                                                       |          |
+---+-------------------------------------------------------+----------+
| 7 | Meet facility short and long term needs to support    | Out      |
| . | mission requirements: critical facilities, including  | standing |
| 1 | nuclear facilities, will meet operational needs for   |          |
|   | programmatic work requirements by minimizing          |          |
| a | unplanned system outages and downtime                 |          |
+---+-------------------------------------------------------+----------+
| 7 | Meet facility short and long term needs to support    | Out      |
| . | mission requirements: achieve the objectives in the   | standing |
| 1 | approved FY 2003 Ten-Year Comprehensive Site Plan     |          |
|   | (TYCSP)                                               |          |
| b |                                                       |          |
+---+-------------------------------------------------------+----------+
| 7 | Achieve continual improvement in ISM:                 | E        |
| . |                                                       | xcellent |
| 2 | • Develop and implement simplified facility safety    |          |
|   | basis and related operational requirements for        |          |
|   | non-nuclear facilities based on benchmarking of best  |          |
|   | practices                                             |          |
|   |                                                       |          |
|   | • Assure consistent application of ISM principles     |          |
|   | across all organization levels and across all         |          |
|   | Laboratory facilities                                 |          |
+---+-------------------------------------------------------+----------+
|   |                                                       | Sati     |
|   |                                                       | sfactory |
+---+-------------------------------------------------------+----------+
| 7 | Comply with 10 CFR 830 subpart B for the operations   | Sati     |
| . | of the Laboratories' category 2 and 3 nuclear         | sfactory |
| 3 | facilities by completing the required Documented      |          |
|   | Safety Analysis and Technical Safety Requirements     |          |
|   | according to the Approved Submission Plan             |          |
+---+-------------------------------------------------------+----------+
| 7 | Complete the NNSA-approved action plans and           | Sati     |
| . | UC-approved project plans for implementing Integrated | sfactory |
| 4 | Safeguards and Security Management and after that,    |          |
|   | achieve continuous improvements by providing          |          |
|   | consistency throughout the Laboratory                 |          |
+---+-------------------------------------------------------+----------+
| 7 | Develop with NNSA a long-term plan to reduce          | E        |
| . | inventories of surplus and excess SNM and onsite      | xcellent |
| 5 | waste                                                 |          |
+---+-------------------------------------------------------+----------+
| 7 | Maintain an Environmental Management Program          | E        |
| . | consistent with DOE negotiated regulatory             | xcellent |
| 6 | requirements and funding levels (LLNL)                |          |
+---+-------------------------------------------------------+----------+
| 7 | Implement an Emergency Management Program within NNSA | Sati     |
| . | approved schedules (LLNL)                             | sfactory |
| 7 |                                                       |          |
+---+-------------------------------------------------------+----------+
| 7 | Ensure effective controls in business systems by      | Out      |
| . | assessing existing controls and, where needed,        | standing |
| 8 | strengthening controls to ensure effective            |          |
|   | stewardship of public assets                          |          |
+---+-------------------------------------------------------+----------+

  -------- ------------------------------------------------------- -----------
  **8.**   Utilize UC strengths to recruit, retain and develop the 
           workforce basis                                         

  8.1      Provide skills necessary to enhance the science base by Excellent
           implementing integrated recruiting and retention        
           strategies to meet the Laboratories' long-range skills  
           requirements                                            

  8.2      Implement leadership and management development         Excellent
           programs aligned with workforce planning and diversity  
           objectives                                              
  -------- ------------------------------------------------------- -----------

  -------- ------------------------------------------------------- -------------
  **9.**   Sustain effective Community Initiatives                 

  9.1      Leveraging the UC expertise and mission in science      Outstanding
           education, the laboratories will establish and maintain 
           science education outreach programs with the joint      
           goals of community outreach and substantive             
           contribution to science education                       

  9.2      The Laboratory will develop local community initiatives Outstanding
           to include those programs or responses addressing       
           mutual goals and concerns (LLNL)                        
  -------- ------------------------------------------------------- -------------

Appendix B

Acronyms Used in This Report

  -------------- --------------------------------------------------------
  CI             Counterintelligence

  DBT            Design Basis Threat

  DHS            Department of Homeland Security

  DOE            U. S. Department of Energy

  DWTF           Decontamination/Waste Treatment Facility (DWTF)

  ETCU           Engineering Technology Complex Upgrade

  FIRP           Facility and Infrastructure Recapitalization Program

  ISM            Integrated Safety Management

  ISSM           Integrated Safeguards and Security Management

  LLNL           Lawrence Livermore National Laboratory

  LSO            Livermore Site Office

  MC&A           Material Control and Accountability

  NIF            National Ignition Facility

  NNSA           National Nuclear Security Administration

  QMU            Quantification of Margins and Uncertainties

  RTBF           Readiness in Technical Base and Facilities

  SAFE           Security Awareness for Employees

  SCIF           Sensitive Compartmented Information Facility

  SECON          Security Condition

  SEMI           Safety and Emergency Preparedness Inspection

  SNM            Special Nuclear Material

  TSF            Terascale Simulation Facility

  TYCSP          Ten Year Comprehensive Site Plan

  UC             University of California
  -------------- --------------------------------------------------------