**STATEMENT OF WORK**

Independent Verification and Validation (IV&V) Services

Table of Contents

[1. Introduction 2](#introduction)

[2. Federal Role In And Perspective On IV&V
2](#__RefHeading___Toc188248058)

[3. Frequency of IV&V Services 3](#frequency-of-ivv-services)

[4. Conflict of Interest Exclusion 4](#conflict-of-interest-exclusion)

[5. Contractor Capability 4](#contractor-capability)

[6. Key Personnel 6](#key-personnel)

[7. Scope of Services 6](#scope-of-services)

[7.1 IV&V Standards 6](#ivv-standards)

[7.2 IV&V Project Management 7](#ivv-project-management)

[7.3 Planning Oversight 8](#planning-oversight)

[7.4 Project Management 8](#project-management)

[7.5 Quality Management 10](#quality-management)

[7.6 Training 11](#training)

[7.7 Requirements Management 12](#requirements-management)

[7.8 Operating Environment 13](#operating-environment)

[7.9 Development Environment 13](#development-environment)

[7.10 Software Development 14](#software-development)

[7.11 System And Acceptance Testing 15](#system-and-acceptance-testing)

[7.12 Data Management 16](#data-management)

[7.13 Operations Oversight 16](#operations-oversight)

[8. Deliverables 17](#deliverables)

[8.1 Description of IV&V Contract Deliverables
17](#description-of-ivv-contract-deliverables)

[9. IV&V Format and Content Reporting Requirements
22](#ivv-format-and-content-reporting-requirements)

[10. State Furnished Items 23](#state-furnished-items)

[11. Travel 24](#travel)

[12. Points of Contact 24](#points-of-contact)

[13. Contract Period 24](#contract-period)

[14. Definition of Terms 24](#definition-of-terms)

# Introduction 

Software Verification and Validation (V&V) is a systems engineering
discipline helping a development organization build quality into the
application software during the software life cycle. Validation is
concerned with checking that the software meets the user\'s needs, and
Verification is concerned with checking that the system is
well-engineered.

The definition of activities included under software V&V is necessarily
quite broad, and includes both technical and management-based
activities. The Federal approach to V&V differs somewhat from the
international standard for software V&V, namely that found in the
*Institute of Electrical and Electronic Engineers (IEEE) Standard for
Software Verification and Validation* (IEEE Std 1012-2004). Contrary to
the international standard, Federal V&V does not require a continuous
on-site presence or extensive testing, nor does it perform actual
quality assurance activities or other remediations. It instead imposes
periodic reviews of software development projects that include site
visits employing various industry standards to conduct artifact analysis
with interviews of a project's team and stakeholder in order to fashion
a comprehensive "snapshot" of a project's management and technical
processes at work at a given point-in-time.

Another distinction the reader will note is in the inclusion of the word
"Independent" in front of Verification and Validation (V&V). In other
words, Independent V&V is the set of verification and validation
activities performed by an agency not under the control of the
organization that is developing the software. IV&V services must be
provided and managed by an organization that is *technically* and
*managerially* *independent* of the subject software development
project. This independence takes two mandatory forms. First, t*echnical
independence* requires that the IV&V services provider not
organizationally be or have been, nor use personnel who are or were,
involved in the software development or implementation effort, or for
that matter participated in the project's initial planning and/or
subsequent design. Such technical independence helps ensure every IV&V
review report is free of personal or professional bias, posturing, or
*goldplating*. Secondly, *managerial independence* is required of the
IV&V services provider to ensure that the IV&V effort is vested in an
organization departmentally and hierarchically separate from the
software development and program management organizations. Such
managerial independence helps ensure that the IV&V service provider is
able to deliver to both State and Federal executive leadership and
management, findings and recommendations of an IV&V review without
restriction, fear of retaliation, or coercion (e.g., reports being
subject to prior review or approval from the development group before
release to outside entities, such as the Federal government.)

This procurement document defines the IV&V services required by \[insert
State Organization Name\] in support of the \[insert Project Name\]
automation effort. .

# Federal Role In And Perspective On IV&V

For Federal purposes, the scope of IV&V has been expanded to include
planning, management, and other programmatic activities in conformance
with the term's usage in Federal regulations at 45 CFR 307.15(b)(10).

As previously stated, offerors to this solicitation need to be aware
that the requirements of IV&V on the State\'s project do not necessarily
conform to industry standard practices for verification and validation
as defined in the IEEE Standard for Software Verification and Validation
(IEEE Std 1012-2004). The Federal requirements for IV&V are, in fact, a
subset of the full V&V standard defined by the IEEE Std 1012-2004. The
IEEE standard describes software V&V processes as generally determining
if development products of a given activity conform to the requirements
of that activity, and if the software satisfies the intended use and
user needs. In other words, the IEEE standard answers the dual question,
"... did we build the product right, and did we build the right
product?" As defined in the IEEE standards, V&V processes include
activities such as assessment, analysis, measurement, inspection, and
testing of software products and processes. These IV&V processes further
include assessing software in the context of the system, including the
operational environment, hardware, interfacing software, operators, and
users. The IEEE standard seeks to assure that software V&V is performed
in parallel with software development, not intermittently or at the
conclusion of the software development.

However, the Federal requirements for IV&V on State automation projects
are limited in their scope from the industry standard IEEE definition
for V&V in two key regards:

1.  IV&V of the project is not considered to be an integral process
    within the larger development project. Rather, it is considered to
    be an adjunct activity that does not fall within the managerial
    oversight or control of the day-to-day operation of the project's
    management structure, including any and all of its \"umbrella\"
    agencies. The IV&V Service Provider must maintain organizational
    independence and autonomy from the project's organization, and
    therefore has a reduced role from that normally associated with full
    IV&V services. In some respects, the IV&V Service Provider can be
    viewed as performing a "Technology Audit."

2.  The IV&V Service Provider shall provide its detailed, structured
    reports of findings of deficiencies and recommendations for their
    remediation to the cognizant Federal Office at the same time as they
    are presented to the State. This reporting process, in accordance
    with Federal regulations, includes not only final report issuance,
    but all draft report submissions as well.

# Frequency of IV&V Services 

The frequency of IV&V oversight services under this procurement,
resulting in a report of findings and recommendations has been
determined to be Semi-Annual. Any offeror whose proposal suggests a
constant presence on or within the \[insert Project name\] will likely
find their costs unnecessarily higher than those of an offeror who
proposes to accomplish the same mission (from IV&V review initiation to
final report delivery and presentation) within the otherwise defined,
periodic timeframe of semi-annual. For purposes of this solicitation, we
believe the offeror\'s periodic IV&V reviews should each take no longer
than an eight to ten-week timeframe from initiation through to final
report delivery and presentation. Further, though an offeror may indeed
find need of multiple disciplines in the conduct of each periodic IV&V
review, great care should be taken in the formulation of its overall
project work plan and proposal not to propose unnecessary layers of
management and contract oversight. From the State's perspective,
excessive management staffing in an offer's IV&V review team is neither
desirable nor appropriate, and should be avoided.

# Conflict of Interest Exclusion

Any contractor (and its subcontractors) serving in the role of IV&V
Service Contractor/Provider to the State \[insert Project name\] Project
is prohibited from soliciting, proposing, or being awarded any project
management, quality assurance, software design, development, or other
manner of planning, design, development, or implementation phase
activity on the subject \[insert Project name\] Project for which these
IV&V services are being procured. This exclusion likewise extends to any
other project within the Department that may interact with or otherwise
provide services to the subject \[insert Project name\] Project or to
the Department during the full term of this contract. This exclusion is
executed in accordance with Federal regulations at 45 CFR Part
307.15(b)(10)(ii), which require that this IV&V effort, \"\... be
conducted by an entity that is independent from the State (unless the
State receives an exception from OCSE).\" For purposes of clarity, OCSE
has defined "the State" in the above regulatory citation as being a
State's IT project, the IV-D agency itself, and the IV-D agency's
umbrella agency or Department. The primary purpose of this exclusion is
to ensure the IV&V Service Provider does not find itself involved with
any real or perceived conflicts of interest. Such conflicts of interest
could be alleged were the IV&V Service Provider found to be reviewing
work products, deliverables, and/or processes for which they currently
are or were responsible to plan, design, develop, implement, or operate.
Therefore, these exclusions seek to ensure the credibility of the IV&V
Service Provider, or in the words of an old colloquialism, to prevent,
"the fox watching the hen house." All exceptions to this conflict of
interest exclusion will require Federal OCSE written approval prior to
any exception being granted to the IV&V Service Provider.

# Contractor Capability

The offeror must have a demonstrated ability to perform the following
activities, which are the same as those stated in Federal regulations at
45 CFR Part 307.15:

a.  Develop a project work plan. The plan must be provided directly to
    the cognizant Federal Office at the same time it is given to the
    State.

b.  Review and make recommendations on both the management of the
    \[insert Project name\] Project, both State and vendor, and the
    technical aspects of the \[insert Project name\] Project. The
    results of this analysis must be provided directly to the cognizant
    Federal Office at the same time it is given to the State.

c.  Consult with all stakeholders and assess the user involvement and
    buy-in regarding system functionality and the system\'s ability to
    meet program needs.

d.  Conduct an analysis of past \[insert Project name\] Project
    performance (schedule, budget) sufficient to identify and make
    recommendations for improvement.

e.  Provide a risk management assessment and capacity planning services.

f.  Develop performance metrics which allow tracking of \[insert Project
    name\] Project completion against milestones set by the State.

g.  The offeror must also possess the corporate knowledge and experience
    demonstrating the following capabilities and capacities:

h.  Develop a project management plan, including recommendations for:
    adequate staff; staff skills, positions and abilities; equipment
    resources; training and facilities; and functional responsibility
    and authority within a structured project organization.

i.  Analyze \[insert Project name\] Project management; evaluate
    \[insert Project name\] Project progress, resources, budget,
    schedules, work flow and reporting.

j.  Review and analyze \[insert Project name\] Project management
    planning documents.

k.  Review and analyze \[insert Project name\] Project software
    development documents.

l.  Review and monitor development processes to ensure they are being
    documented, carried out, and analyzed for improvement.

m.  Assess the \[insert Project name\] Project's Configuration
    Management (CM) function/ organization by reviewing CM reports and
    making recommendations regarding appropriate processes and tools to
    manage system changes.

n.  Perform a detailed review of \[insert Project name\] Project
    deliverables for accuracy, completeness, and adherence to
    contractual and functional requirements.

o.  Perform a detailed review of the system documentation (Requirements,
    Design, Training, Test, and Management Plans, etc.) for accuracy and
    completeness.

p.  Perform a detailed review of the software architecture for
    feasibility, consistency, and adherence to industry standards.

q.  Inventory and review the application software for completeness and
    adherence to programming standards for the \[insert Project name\]
    Project.

r.  Analyze application, network, hardware and software operating
    platform performance characteristics relative to
    expected/anticipated/contractually guaranteed results and industry
    standards/expectations.

s.  Review the process for tracking of business and technical
    requirements to their source and review the process established
    during the planning phase for requirements traceability throughout
    the subsequent development/implementation phase. Review the
    traceability of system requirements to design, code, test, and
    training.

t.  Assess and recommend improvement, as needed, to assure maintenance
    of a data center, including data center input to the \[insert
    Project name\] Project regarding operational and maintenance
    performance of the application.

u.  Assess and recommend improvement, as needed, to assure software
    testing is being performed adequately through review of test plans
    or other documentation and through direct observation of testing
    where appropriate, including participation in and coordination of
    peer reviews.

v.  Assess and recommend improvement, as needed, to assure appropriate
    user and developer training is planned and carried out.

w.  Review system hardware and software configuration and report on any
    compatibility and obsolescence issues.

x.  Review and analyze system capacity studies.

# Key Personnel

Each proposal for IV&V services must include a resume with the
experience and skills of the key personnel proposed for the IV&V Service
Provider contract. For purposes of this solicitation, all contractor
staff supplying services to this IV&V contract are key personnel. In
addition to providing resumes for all key personnel, each proposal for
IV&V services must also specify by name, the position descriptions,
titles, and areas of responsibility of the IV&V personnel who actually
will work on the \[insert Project name\] Project.

The contractor and the State agree that the key personnel are critical
to the performance of the contract and, therefore, the State has the
right of refusal for any personnel replacements, substitutions, or
reassignments of duties of key personnel assigned to the IV&V services
contract. The State will also be notified, in writing, of any requests
for changes to the personnel assigned to the IV&V contract tasks.
Likewise, after contract award, the IV&V provider shall secure written
approval from the State prior to making any changes to key personnel. In
all instances, qualifications for suggested staff changes should be
comparable with those being replaced. Finally, all offerors to this
solicitation must be aware that the State must submit key personnel
information to the cognizant Federal Office for their review and
approval of those contractor's key personnel to this contract prior to
contract award, and that all subsequent personnel changes may require
prior Federal review and approval, and that these approvals are in
addition to any State approvals.

# Scope of Services

Using pre-defined checklists and similar tools founded on industry
standards, the IV&V Service Provider staff will interview and observe
\[insert Project name\] Project Management staff, CSE Program staff, the
\[insert Project name\] Project Development Contractor staff (including
any sub-contractors), observe project meetings and activities to
understand the processes, procedures, and tools used in the CSE Program
and \[insert Project name\] Project environments, and review and analyze
for adherence to accepted, contractually-defined industry standards, all
applicable and available documentation. As a result of these
interactions and reviews of the applicable \[insert Project name\]
Project documentation, the IV&V Service Provider will produce a
structured, exception-based quarterly assessment report that objectively
illustrates the strengths and weaknesses of the Project. The IV&V
Service Provider will also provide recommendations for correcting the
weaknesses that the assessment reports identify.

To ensure the independence of the IV&V effort, all deliverables will be
submitted concurrently to OCSE when a copy is transmitted to the
cognizant State Contract Manager. This includes all workplans, review
checklists, Deliverables Observation Review (DOR) reports, and draft and
final Quarterly Review (QR) reports. Final documents will likewise be
delivered to OCSE by the IV&V Service Provider at the same time that
they are submitted to the Department and agency.

## IV&V Standards {#ivv-standards .Style-Heading-2-+-Not-Bold}

Applicable tasks and activities will be performed in accordance with the
Institute of Electrical and Electronics Engineers (IEEE) Standard
1012-2004. The IV&V Service Provider will also use all other applicable,
lifecycle-appropriate IEEE Standards (e.g., 12207 Software Life Cycle
Process; 703 Software QA Plans; 1074 Developing Software Project
Lifecycle Process; 828 Configuration Management Plans; and, 830
Requirement Specifications, etc., to name a few) in assessing the
State's \[insert Project name\] Project. Further, the IV&V Service
Provider will employ the Capability Maturity Model Integrated (CMMI),
and the Project Management Institute's Project Management Body of
Knowledge (PMBOK) Third Edition, and the PMBOK - Government Extension,
as additional standards by which to assess the \[insert Project name\]
Project. Offerors to this contract must clearly and thoroughly describe
in their technical response, their approach to using, at a minimum,
these three industry standards (CMMI, PMBOK, IEEE) Where an offeror has
a similar, corresponding, but different set of minimum standards than
those cited above, the offeror will be expected to cross-reference or
otherwise map how their own standards meet the same level of detail and
scope of review as the industry standards for IV&V cited herein (e.g.,
CMMI, PMBOK and IEEE.) Failure to do provide this cross-referencing of
standards in the offeror's proposal will be deemed as being
non-responsive to this solicitation for purposes of evaluation of the
offeror's proposal.

IV&V services will be performed periodically, through performance of
semi-annual IV&V reviews, as part of a larger oversight role of the
day-to-day operations and management of the \[insert Project name\]
Project by State and Federal entities. To support the IV&V Service
Provider in this role in a timely manner, the IV&V Service Provider
shall have complete access to \[insert Project name\] Project documents,
facilities, and staff during normal business hours as required to carry
out their oversight role. The IV&V Service Provider shall have access to
all key staff on site at the \[insert Project name\] Project location(s)
daily, as needed to observe meetings, review deliverables and
documentation, conduct interviews, etc., in order to ensure a high level
of integrity and confidence in the IV&V Service Provider's \[insert
Project name\] Project oversight and monitoring.

The following section contains lists of individual IV&V activities. All
activities in Sections 7.2 through 7.13 are mandatory IV&V activities
and considered part of this solicitation. The checked activities should
be costed and scheduled in the offeror's IV&V Project Management Plan
and reported on in the Initial and Periodic IV&V Reports.

## IV&V Project Management {#ivv-project-management .Style-Heading-2-+-Not-Bold}

  --------------------------- ------------------- -----------------------------------------------------
  **[IV&V Project                                 
  Management]{.smallcaps}**                       

  **[Task Item]{.smallcaps}** **[Task             **[Task Description]{.smallcaps}**
                              \#]{.smallcaps}**   

  **IV&V Management Plan**    **IM-1**            As the first deliverable the IV&V provider shall
                                                  develop an IV&V Management Plan. This plan shall
                                                  describe the activities, personnel, schedule,
                                                  standards, and methodology for conducting the IV&V
                                                  reviews. (see *Deliverables* for more details)

  **Conduct Initial Review**  **IM-2**            Prepare and deliver an Initial IV&V report on the
                                                  required activities. Report on status of each
                                                  activity. (see *Deliverables* for more details)

  **Conduct Periodic          **IM-3**            Prepare and deliver a Follow-up IV&V report on the
  Review(s)**                                     required activities. Report on status of each
                                                  activity and progress since the previous report. (see
                                                  *Deliverables* for more details)

  **Management Briefing**     **IM-4**            Prepare and deliver a formal presentation(s) on the
                                                  status of the IV&V project. Presented as required,
                                                  with at least ten (10) business days notice. No more
                                                  than once a month. (see *Deliverables* for more
                                                  details)
  --------------------------- ------------------- -----------------------------------------------------

## Planning Oversight

  --------------------- ------------------- -----------------------------------------------------
  **[Task               **[Task             **[Task Description]{.smallcaps}**
  Item]{.smallcaps}**   \#]{.smallcaps}**   

  **Procurement**       **PO-1**            Verify the procurement strategy supports State and
                                            Federal project objectives.

                        **PO-2**            Review and make recommendations on the solicitation
                                            documents relative to their ability to adequately
                                            inform potential vendors about project objectives,
                                            requirements, risks, etc.

                        **PO-3**            Verify the evaluation criteria are consistent with
                                            project objectives and evaluation processes are
                                            consistently applied; verify all evaluation criteria
                                            is metrics based and clearly articulated within the
                                            solicitation documents.

                        **PO-4**            Verify that the obligations of the vendor,
                                            sub-contractors and external staff (terms,
                                            conditions, statement of work, requirements,
                                            technical standards, performance standards,
                                            development milestones, acceptance criteria, delivery
                                            dates, etc.) are clearly defined. This includes
                                            verifying that performance metrics have been included
                                            that will allow tracking of project performance and
                                            progress against criteria set by the State.

                        **PO-5**            Verify the final contract for the vendor team states
                                            that the vendor will participate in the IV&V process,
                                            being cooperative for coordination and communication
                                            of information.

  **Feasibility Study** **PO-6**            Perform ongoing assessment and review of State
                                            methodologies used for the feasibility study,
                                            verifying it was objective, reasonable, measurable,
                                            repeatable, consistent, accurate and verifiable.

                        **PO-7**            Review and evaluate the PAPD(U)/IAPD(U) documents.

                        **PO-8**            Review and evaluate the Cost Benefit Analysis to
                                            assess its reasonableness.
  --------------------- ------------------- -----------------------------------------------------

## Project Management {#project-management .Style-Heading-2-+-Not-Bold}

  --------------------- ------------------- -----------------------------------------------------
  **[Task               **[Task             **[Task Description]{.smallcaps}**
  Item]{.smallcaps}**   \#]{.smallcaps}**   

  **Project             **PM-1**            Assess and recommend improvement, as needed, to
  Sponsorship**                             assure continuous executive stakeholder buy-in,
                                            participation, support and commitment, and that open
                                            pathways of communication exist among all
                                            stakeholders.

                        **PM-2**            Verify that executive sponsorship has bought-in to
                                            all changes which impact project objectives, cost, or
                                            schedule.

  **Management          **PM-3**            Verify and assess project management and
  Assessment**                              organization, verify that lines of reporting and
                                            responsibility provide adequate technical and
                                            managerial oversight of the project.

                        **PM-4**            Evaluate project progress, resources, budget,
                                            schedules, work flow, and reporting.

                        **PM-5**            Assess coordination, communication and management to
                                            verify agencies and departments are not working
                                            independently of one another and following the
                                            communication plan.

  **Project             **PM-6**            Verify that a Project Management Plan is created and
  Management**                              being followed. Evaluate the project management plans
                                            and procedures to verify that they are developed,
                                            communicated, implemented, monitored and complete.

                        **PM-7**            Evaluate project reporting plan and actual project
                                            reports to verify project status is accurately traced
                                            using project metrics.

                        **PM-8**            Verify milestones and completion dates are planned,
                                            monitored, and met.

                        **PM-9**            Verify the existence and institutionalization of an
                                            appropriate project issue tracking mechanism that
                                            documents issues as they arise, enables communication
                                            of issues to proper stakeholders, documents a
                                            mitigation strategy as appropriate, and tracks the
                                            issue to closure. This should include but is not
                                            limited to technical and development efforts.

                        **PM-10**           Evaluate the system's planned life-cycle development
                                            methodology or methodologies (waterfall, evolutionary
                                            spiral, rapid prototyping, incremental, etc.) to see
                                            if they are appropriate for the system being
                                            developed.

  **Business Process    **PM-12**           Evaluate the project's ability and plans to redesign
  Reengineering**                           business systems to achieve improvements in critical
                                            measures of performance, such as cost, quality,
                                            service, and speed.

                        **PM-13**           Verify that the reengineering plan has the strategy,
                                            management backing, resources, skills and incentives
                                            necessary for effective change.

                        **PM-14**           Verify that resistance to change is anticipated and
                                            prepared for by using principles of change management
                                            at each step (such as excellent communication,
                                            participation, incentives) and having the appropriate
                                            leadership (executive pressure, vision, and actions)
                                            throughout the reengineering process.

  **Risk Management**   **PM-15**           Verify that a Project Risk Management Plan is created
                                            and being followed. Evaluate the projects risk
                                            management plans and procedures to verify that risks
                                            are identified and quantified and that mitigation
                                            plans are developed, communicated, implemented,
                                            monitored, and complete.

  **Change Management** **PM-16**           Verify that a Change Management Plan is created and
                                            being followed. Evaluate the change management plans
                                            and procedures to verify they are developed,
                                            communicated, implemented, monitored, and complete;
                                            and that resistance to change is anticipated and
                                            prepared for.

  **Communication       **PM-17**           Verify that a Communication Plan is created and being
  Management**                              followed. Evaluate the communication plans and
                                            strategies to verify they support communications and
                                            work product sharing between all project
                                            stakeholders; and assess if communication plans and
                                            strategies are effective, implemented, monitored and
                                            complete.

  **Configuration       **PM-18**           Review and evaluate the configuration management (CM)
  Management**                              plans and procedures associated with the development
                                            process.

                        **PM-19**           Verify that all critical development documents,
                                            including but not limited to requirements, design,
                                            code and JCL are maintained under an appropriate
                                            level of control.

                        **PM-20**           Verify that the processes and tools are in place to
                                            identify code versions and to rebuild system
                                            configurations from source code.

                        **PM-21**           Verify that appropriate source and object libraries
                                            are maintained for training, test, and production and
                                            that formal sign-off procedures are in place for
                                            approving deliverables.

                        **PM-22**           Verify that appropriate processes and tools are in
                                            place to manage system changes, including formal
                                            logging of change requests and the review,
                                            prioritization and timely scheduling of maintenance
                                            actions.

                        **PM-23**           Verify that mechanisms are in place to prevent
                                            unauthorized changes being made to the system and to
                                            prevent authorized changes from being made to the
                                            wrong version.

                        **PM-24**           Review the use of CM information (such as the number
                                            and type of corrective maintenance actions over time)
                                            in project management.

  **Project Estimating  **PM-25**           Evaluate and make recommendations on the estimating
  and Scheduling**                          and scheduling process of the project to ensure that
                                            the project budget and resources are adequate for the
                                            work-breakdown structure and schedule.

                        **PM-26**           Review schedules to verify that adequate time and
                                            resources are assigned for planning, development,
                                            review, testing and rework.

                        **PM-27**           Examine historical data to determine if the
                                            project/department has been able to accurately
                                            estimate the time, labor and cost of software
                                            development efforts.

  **Project Personnel** **PM-28**           Examine the job assignments, skills, training and
                                            experience of the personnel involved in program
                                            development to verify that they are adequate for the
                                            development task.

                        **PM-29**           Evaluate the State's hiring plan for the project to
                                            verify that adequate human resources will be
                                            available for development and maintenance.

                        **PM-30**           Evaluate the State's personnel policies to verify
                                            that staff turnover will be minimized.

  **Project             **PM-31**           Verify that lines of reporting and responsibility
  Organization**                            provide adequate technical and managerial oversight
                                            of the project.

                        **PM-32**           Verify that the project's organizational structure
                                            supports training, process definition, independent
                                            Quality Assurance, Configuration Management, product
                                            evaluation, and any other functions critical for the
                                            projects success.

  **Subcontractors and  **PM-33**           Evaluate the use of sub-contractors or other external
  External Staff**                          sources of project staff (such as IS staff from
                                            another State organization) in project development.

                        **PM-34**           Verify that the obligations of sub-contractors and
                                            external staff (terms, conditions, statement of work,
                                            requirements, standards, development milestones,
                                            acceptance criteria, delivery dates, etc.) are
                                            clearly defined.

                        **PM-35**           Verify that the subcontractors' software development
                                            methodology and product standards are compatible with
                                            the system's standards and environment.

                        **PM-36**           Verify that the subcontractor has and maintains the
                                            required skills, personnel, plans, resources,
                                            procedures and standards to meet their commitment.
                                            This will include examining the feasibility of any
                                            offsite support of the project

                        **PM-37**           Verify that any proprietary tools used by
                                            subcontractors do not restrict the future
                                            maintainability, portability, and reusability of the
                                            system.

  **State Oversight**   **PM-38**           Verify that State oversight is provided in the form
                                            of periodic status reviews and technical
                                            interchanges.

                        **PM-39**           Verify that the State has defined the technical and
                                            managerial inputs the subcontractor needs (reviews,
                                            approvals, requirements and interface clarifications,
                                            etc.) and has the resources to supply them on
                                            schedule.

                        **PM-40**           Verify that State staff has the ultimate
                                            responsibility for monitoring project cost and
                                            schedule.
  --------------------- ------------------- -----------------------------------------------------

## Quality Management {#quality-management .Style-Heading-2-+-Not-Bold}

  --------------------- ------------------- -----------------------------------------------------
  **[Task               **[Task             **[Task Description]{.smallcaps}**
  Item]{.smallcaps}**   \#]{.smallcaps}**   

  **Quality Assurance** **QA-1**            Evaluate and make recommendations on the project's
                                            Quality Assurance plans, procedures and organization.

                        **QA-2**            Verify that QA has an appropriate level of
                                            independence from project management.

                        **QA-3**            Verify that the QA organization monitors the fidelity
                                            of all defined processes in all phases of the
                                            project.

                        **QA-4**            Verify that the quality of all products produced by
                                            the project is monitored by formal reviews and
                                            sign-offs.

                        **QA-5**            Verify that project self-evaluations are performed
                                            and that measures are continually taken to improve
                                            the process.

                        **QA-6**            Monitor the performance of the QA contractor by
                                            reviewing its processes and reports and performing
                                            spot checks of system documentation; assess findings
                                            and performance of the processes and reports.

                        **QA-7**            Verify that QA has an appropriate level of
                                            independence; evaluate and make recommendations on
                                            the project's Quality Assurance plans, procedures and
                                            organization.

                        **QA-8**            Verify that the QA vendor provides periodic
                                            assessment of the CMM activities of the project and
                                            that the project takes action to reach and maintain
                                            CMM Level \_\_.

                        **QA-9**            Evaluate if appropriate mechanisms are in place for
                                            project self-evaluation and process improvement.

  **Process Definition  **QA-10**           Review and make recommendations on all defined
  and Product                               processes and product standards associated with the
  Standards**                               system development.

                        **QA-11**           Verify that all major development processes are
                                            defined and that the defined and approved processes
                                            and standards are followed in development.

                        **QA-12**           Verify that the processes and standards are
                                            compatible with each other and with the system
                                            development methodology.

                        **QA-13**           Verify that all process definitions and standards are
                                            complete, clear, up-to-date, consistent in format,
                                            and easily available to project personnel
  --------------------- ------------------- -----------------------------------------------------

## Training {#training .Style-Heading-2-+-Not-Bold}

  --------------------- ------------------- -----------------------------------------------------
  **[Task               **[Task             **[Task Description]{.smallcaps}**
  Item]{.smallcaps}**   \#]{.smallcaps}**   

  **User Training and   **TR-1**            Review and make recommendations on the training
  Documentation**                           provided to system users. Verify sufficient knowledge
                                            transfer for maintenance and operation of the new
                                            system.

                        **TR-2**            Verify that training for users is instructor-led and
                                            hands-on and is directly related to the business
                                            process and required job skills.

                        **TR-3**            Verify that user-friendly training materials and help
                                            desk services are easily available to all users.

                        **TR-4**            Verify that all necessary policy and process and
                                            documentation is easily available to users.

                        **TR-5**            Verify that all training is given on-time and is
                                            evaluated and monitored for effectiveness, with
                                            additional training provided as needed.

  **Developer Training  **TR-6**            Review and make recommendations on the training
  and Documentation**                       provided to system developers.

                        **TR-7**            Verify that developer training is technically
                                            adequate, appropriate for the development phase, and
                                            available at appropriate times.

                        **TR-8**            Verify that all necessary policy, process and
                                            standards documentation is easily available to
                                            developers.

                        **TR-9**            Verify that all training is given on-time and is
                                            evaluated and monitored for effectiveness, with
                                            additional training provided as needed.
  --------------------- ------------------- -----------------------------------------------------

## Requirements Management {#requirements-management .Style-Heading-2-+-Not-Bold}

  --------------------- ------------------- -----------------------------------------------------
  **[Task               **[Task             **[Task Description]{.smallcaps}**
  Item]{.smallcaps}**   \#]{.smallcaps}**   

  **Requirements        **RM-1**            Evaluate and make recommendations on the project's
  Management**                              process and procedures for managing requirements.

                        **RM-2**            Verify that system requirements are well-defined,
                                            understood and documented.

                        **RM-3**            Evaluate the allocation of system requirements to
                                            hardware and software requirements.

                        **RM-4**            Verify that software requirements can be traced
                                            through design, code and test phases to verify that
                                            the system performs as intended and contains no
                                            unnecessary software elements.

                        **RM-5**            Verify that requirements are under formal
                                            configuration control.

  **Security            **RM-6**            Evaluate and make recommendations on project policies
  Requirements**                            and procedures for ensuring that the system is secure
                                            and that the privacy of client data is maintained.

                        **RM-7**            Evaluate the projects restrictions on system and data
                                            access.

                        **RM-8**            Evaluate the projects security and risk analysis.

                        **RM-9**            Verify that processes and equipment are in place to
                                            back up client and project data and files and archive
                                            them safely at appropriate intervals.

  **Requirements        **RM-10**           Verify that an analysis of client, State and federal
  Analysis**                                needs and objectives has been performed to verify
                                            that requirements of the system are well understood,
                                            well defined, and satisfy federal regulations.

                        **RM-11**           Verify that all stakeholders have been consulted to
                                            the desired functionality of the system, and that
                                            users have been involved in prototyping of the user
                                            interface.

                        **RM-12**           Verify that all stakeholders have bought-in to all
                                            changes which impact project objectives, cost, or
                                            schedule.

                        **RM-13**           Verify that performance requirements (e.g. timing,
                                            response time and throughput) satisfy user needs

                        **RM-14**           Verify that user's maintenance requirements for the
                                            system are completely specified

  **Interface           **RM-15**           Verify that all system interfaces are exactly
  Requirements**                            described, by medium and by function, including
                                            input/output control codes. data format, polarity,
                                            range, units, and frequency.

                        **RM-16**           Verify those approved interface documents are
                                            available and that appropriate relationships (such as
                                            interface working groups) are in place with all
                                            agencies and organizations supporting the interfaces.

  **Requirements        **RM-17**           Verify that all system requirements have been
  Allocation and                            allocated to a either a software or hardware
  Specification**                           subsystem.

                        **RM-18**           Verify that requirements specifications have been
                                            developed for all hardware and software subsystems in
                                            a sufficient level of detail to ensure successful
                                            implementation.

  **Reverse             **RM-19**           If a legacy system or a transfer system is or will be
  Engineering**                             used in development, Verify that a well defined plan
                                            and process for reengineering the system is in place
                                            and is followed. The process, depending on the goals
                                            of the reuse/transfer, may include reverse
                                            engineering, code translation, re-documentation,
                                            restructuring, normalization, and re-targeting.
  --------------------- ------------------- -----------------------------------------------------

## Operating Environment {#operating-environment .Style-Heading-2-+-Not-Bold}

  --------------------- ------------------- -----------------------------------------------------
  **[Task               **[Task             **[Task Description]{.smallcaps}**
  Item]{.smallcaps}**   \#]{.smallcaps}**   

  **System Hardware**   **OE-1**            Evaluate new and existing system hardware
                                            configurations to determine if their performance is
                                            adequate to meet existing and proposed system
                                            requirements.

                        **OE-2**            Determine if hardware is compatible with the State's
                                            existing processing environment, if it is
                                            maintainable, and if it is easily upgradeable. This
                                            evaluation will include, but is not limited to CPUs
                                            and other processors, memory, network connections and
                                            bandwidth, communication controllers,
                                            telecommunications systems (LAN/WAN), terminals,
                                            printers and storage devices.

                        **OE-3**            Evaluate current and projected vendor support of the
                                            hardware, as well as the State's hardware
                                            configuration management plans and procedures.

  **System Software**   **OE-4**            Evaluate new and existing system software to
                                            determine if its capabilities are adequate to meet
                                            existing and proposed system requirements.

                        **OE-5**            Determine if the software is compatible with the
                                            State's existing hardware and software environment,
                                            if it is maintainable, and if it is easily
                                            upgradeable. This evaluation will include, but is not
                                            limited to, operating systems, middleware, and
                                            network software including communications and
                                            file-sharing protocols.

                        **OE-6**            Current and projected vendor support of the software
                                            will also be evaluated, as well as the States
                                            software acquisition plans and procedures.

  **Database Software** **OE-7**            Evaluate new and existing database products to
                                            determine if their capabilities are adequate to meet
                                            existing and proposed system requirements.

                        **OE-8**            Determine if the database's data format is easily
                                            convertible to other formats, if it supports the
                                            addition of new data items, if it is scaleable, if it
                                            is easily refreshable and if it is compatible with
                                            the State's existing hardware and software, including
                                            any on-line transaction processing (OLTP)
                                            environment.

                        **OE-9**            Evaluate any current and projected vendor support of
                                            the software, as well as the State's software
                                            acquisition plans and procedures.

  **System Capacity**   **OE-10**           Evaluate the existing processing capacity of the
                                            system and verify that it is adequate for current
                                            statewide needs for both batch and on-line
                                            processing.

                        **OE-11**           Evaluate the historic availability and reliability of
                                            the system including the frequency and criticality of
                                            system failure.

                        **OE-12**           Evaluate the results of any volume testing or stress
                                            testing.

                        **OE-13**           Evaluate any existing measurement and capacity
                                            planning program and will evaluate the system's
                                            capacity to support future growth.

                        **OE-14**           Make recommendations on changes in processing
                                            hardware, storage, network systems, operating
                                            systems, COTS software, and software design to meet
                                            future growth and improve system performance.
  --------------------- ------------------- -----------------------------------------------------

## Development Environment {#development-environment .Style-Heading-2-+-Not-Bold}

  --------------------- ------------------- -----------------------------------------------------
  **[Task               **[Task             **[Task Description]{.smallcaps}**
  Item]{.smallcaps}**   \#]{.smallcaps}**   

  **Development         **DE-1**            Evaluate new and existing development hardware
  Hardware**                                configurations to determine if their performance is
                                            adequate to meet the needs of system development.

                        **DE-2**            Determine if hardware is maintainable, easily
                                            upgradeable, and compatible with the State's existing
                                            development and processing environment. This
                                            evaluation will include, but is not limited to CPUs
                                            and other processors, memory, network connections and
                                            bandwidth, communication controllers,
                                            telecommunications systems (LAN/WAN), terminals,
                                            printers and storage devices.

                        **DE-3**            Current and projected vendor support of the hardware
                                            will also be evaluated, as well as the State's
                                            hardware configuration management plans and
                                            procedures.

  **Development         **DE-4**            Evaluate new and existing development software to
  Software**                                determine if its capabilities are adequate to meet
                                            system development requirements.

                        **DE-5**            Determine if the software is maintainable, easily
                                            upgradeable, and compatible with the State's existing
                                            hardware and software environment.

                        **DE-6**            Evaluate the environment as a whole to see if it
                                            shows a degree of integration compatible with good
                                            development. This evaluation will include, but is not
                                            limited to, operating systems, network software, CASE
                                            tools, project management software, configuration
                                            management software, compilers, cross-compilers,
                                            linkers, loaders, debuggers, editors, and reporting
                                            software.

                        **DE-7**            Language and compiler selection will be evaluated
                                            with regard to portability and reusability (ANSI
                                            standard language, non-standard extensions, etc.)

                        **DE-8**            Current and projected vendor support of the software
                                            will also be evaluated, as well as the States
                                            software acquisition plans and procedures.
  --------------------- ------------------- -----------------------------------------------------

## Software Development {#software-development .Style-Heading-2-+-Not-Bold}

  --------------------- ------------------- -----------------------------------------------------
  **[Task               **[Task             **[Task Description]{.smallcaps}**
  Item]{.smallcaps}**   \#]{.smallcaps}**   

  **High-Level Design** **SD-1**            Evaluate and make recommendations on existing high
                                            level design products to verify the design is
                                            workable, efficient, and satisfies all system and
                                            system interface requirements.

                        **SD-2**            Evaluated the design products for adherence to the
                                            project design methodology and standards.

                        **SD-3**            Evaluate the design and analysis process used to
                                            develop the design and make recommendations for
                                            improvements. Evaluate design standards, methodology
                                            and CASE tools used will be evaluated and make
                                            recommendations.

                        **SD-4**            Verify that design requirements can be traced back to
                                            system requirements.

                        **SD-5**            Verify that all design products are under
                                            configuration control and formally approved before
                                            detailed design begins.

  **Detailed Design**   **SD-6**            Evaluate and make recommendations on existing
                                            detailed design products to verify that the design is
                                            workable, efficient, and satisfies all high level
                                            design requirements.

                        **SD-7**            The design products will also be evaluated for
                                            adherence to the project design methodology and
                                            standards.

                        **SD-8**            The design and analysis process used to develop the
                                            design will be evaluated and recommendations for
                                            improvements made.

                        **SD-9**            Design standards, methodology and CASE tools used
                                            will be evaluated and recommendations made.

                        **SD-10**           Verify that design requirements can be traced back to
                                            system requirements and high level design.

                        **SD-11**           Verify that all design products are under
                                            configuration control and formally approved before
                                            coding begins.

  **Job Control**       **SD-12**           Perform an evaluation and make recommendations on
                                            existing job control and on the process for designing
                                            job control.

                        **SD-13**           Evaluate the system's division between batch and
                                            on-line processing with regard to system performance
                                            and data integrity.

                        **SD-14**           Evaluate batch jobs for appropriate scheduling,
                                            timing and internal and external dependencies.

                        **SD-15**           Evaluate the appropriate use of OS scheduling
                                            software.

                        **SD-16**           Verify that job control language scripts are under an
                                            appropriate level of configuration control.

  **Code**              **SD-17**           Evaluate and make recommendations on the standards
                                            and process currently in place for code development.

                        **SD-18**           Evaluate the existing code base for portability and
                                            maintainability, taking software metrics including
                                            but not limited to modularity, complexity and source
                                            and object size.

                        **SD-19**           Code documentation will be evaluated for quality,
                                            completeness (including maintenance history) and
                                            accessibility.

                        **SD-20**           Evaluate the coding standards and guidelines and the
                                            projects compliance with these standards and
                                            guidelines. This evaluation will include, but is not
                                            limited to, structure, documentation, modularity,
                                            naming conventions and format.

                        **SD-21**           Verify that developed code is kept under appropriate
                                            configuration control and is easily accessible by
                                            developers.

                        **SD-22**           Evaluate the project's use of software metrics in
                                            management and quality assurance.

  **Unit Test**         **SD-23**           Evaluate the plans, requirements, environment, tools,
                                            and procedures used for unit testing system modules.

                        **SD-24**           Evaluate the level of test automation, interactive
                                            testing and interactive debugging available in the
                                            test environment.

                        **SD-25**           Verify that an appropriate level of test coverage is
                                            achieved by the test process, that test results are
                                            verified, that the correct code configuration has
                                            been tested, and that the tests are appropriately
                                            documented.
  --------------------- ------------------- -----------------------------------------------------

## System And Acceptance Testing {#system-and-acceptance-testing .Style-Heading-2-+-Not-Bold}

  --------------------- ------------------- -----------------------------------------------------
  **[Task               **[Task             **[Task Description]{.smallcaps}**
  Item]{.smallcaps}**   \#]{.smallcaps}**   

  **System Integration  **ST-1**            Evaluate the plans, requirements, environment, tools,
  Test**                                    and procedures used for integration testing of system
                                            modules.

                        **ST-2**            Evaluate the level of automation and the availability
                                            of the system test environment.

                        **ST-3**            Verify that an appropriate level of test coverage is
                                            achieved by the test process, that test results are
                                            verified, that the correct code configuration has
                                            been tested, and that the tests are appropriately
                                            documented, including formal logging of errors found
                                            in testing.

                        **ST-4**            Verify that the test organization has an appropriate
                                            level of independence from the development
                                            organization.

  **Pilot Test**        **ST-5**            Evaluate the plans, requirements, environment, tools,
                                            and procedures for pilot testing the system.

                        **ST-6**            Verify that a sufficient number and type of case
                                            scenarios are used to ensure comprehensive but
                                            manageable testing and that tests are run in a
                                            realistic, real-time environment.

                        **ST-7**            Verify that test scripts are complete, with
                                            step-by-step procedures, required pre-existing events
                                            or triggers, and expected results.

                        **ST-8**            Verify that test results are verified, that the
                                            correct code configuration has been used, and that
                                            the tests runs are appropriately documented,
                                            including formal logging of errors found in testing.

                        **ST-9**            Verify that the test organization has an appropriate
                                            level of independence from the development
                                            organization.

  **Interface Testing** **ST-10**           Evaluate interface testing plans and procedures for
                                            compliance with industry standards.

  **Acceptance and      **ST-11**           Acceptance procedures and acceptance criteria for
  Turnover**                                each product must be defined, reviewed, and approved
                                            prior to test and the results of the test must be
                                            documented. Acceptance procedures must also address
                                            the process by which any software product that does
                                            not pass acceptance testing will be corrected.

                        **ST-12**           Verify that appropriate acceptance testing based on
                                            the defined acceptance criteria is performed
                                            satisfactorily before acceptance of software
                                            products.

                        **ST-13**           Verify that the acceptance test organization has an
                                            appropriate level of independence from the
                                            subcontractor.

                        **ST-14**           Verify that training in using the contractor-supplied
                                            software is be on-going throughout the development
                                            process, especially If the software is to be turned
                                            over to State staff for operation.

                        **ST-15**           Review and evaluate implementation plan.
  --------------------- ------------------- -----------------------------------------------------

## Data Management {#data-management .Style-Heading-2-+-Not-Bold}

  --------------------- ------------------- -----------------------------------------------------
  **[Task               **[Task             **[Task Description]{.smallcaps}**
  Item]{.smallcaps}**   \#]{.smallcaps}**   

  **Data Conversion**   **DM-1**            Evaluate the State's existing and proposed plans,
                                            procedures and software for data conversion.

                        **DM-2**            Verify that procedures are in place and are being
                                            followed to review the completed data for
                                            completeness and accuracy and to perform data
                                            clean-up as required.

                        **DM-3**            Determine conversion error rates and if the error
                                            rates are manageable.

                        **DM-4**            Make recommendations on making the conversion process
                                            more efficient and on maintaining the integrity of
                                            data during the conversion.

  **Database Design**   **DM-5**            Evaluate new and existing database designs to
                                            determine if they meet existing and proposed system
                                            requirements.

                        **DM-6**            Recommend improvements to existing designs to improve
                                            data integrity and system performance.

                        **DM-7**            Evaluate the design for maintainability, scalability,
                                            refreshability, concurrence, normalization (where
                                            appropriate) and any other factors affecting
                                            performance and data integrity.

                        **DM-8**            Evaluate the project's process for administering the
                                            database, including backup, recovery, performance
                                            analysis and control of data item creation.
  --------------------- ------------------- -----------------------------------------------------

## Operations Oversight {#operations-oversight .Style-Heading-2-+-Not-Bold}

  --------------------- ------------------- ---------------------------------------------------
  **[Task               **[Task             **[Task Description]{.smallcaps}**
  Item]{.smallcaps}**   \#]{.smallcaps}**   

  **Operational Change  **OO-1**            Evaluate statewide system's change request and
  Tracking**                                defect tracking processes.

                        **OO-2**            Evaluate implementation of the process activities
                                            and request volumes to determine if processes are
                                            effective and are being followed.

  **Customer & User     **OO-3**            Evaluate user satisfaction with system to determine
  Operational                               areas for improvement
  Satisfaction**                            

  **Operational Goals** **OO-4**            Evaluate impact of system on program goals and
                                            performance standards.

  **Operational         **OO-5**            Evaluate operational plans and processes.
  Documentation**                           

  **Operational         **OO-6**            Evaluate implementation of the process activities
  Processes and                             including backup, disaster recovery and day-to-day
  Activity**                                operations to verify the processes are being
                                            followed.
  --------------------- ------------------- ---------------------------------------------------

# Deliverables

The following table identifies the anticipated deliverables. The State
reserves the right to request additional analyses, as needed. Likewise,
the IV&V Service Provider may suggest development of additional
deliverables in specific areas. The State must authorize the need for
any additional deliverables prior to their development.

Where applicable, the deliverable must be developed in accordance with
CMMI, PMBOK, and IEEE (or substantially and acceptably similar)
standards. When no applicable standard exists, the methodology and
processes used in the analysis and creation of the deliverable must be
delivered to the Federal OCSE and the State prior to its use, and
described in the final deliverable. All deliverables, standards,
processes, plans, and applicable reference materials will be made
available upon request of the State.

Copies of [all]{.underline} deliverables will be delivered
simultaneously to the State and to the Federal OCSE. Frequencies of
deliverables are provided in the table below. The State reserves the
right to extend the due date if appropriate, due to document size,
schedule or changes in scope. The IV&V Service Provider must notify the
State of an anticipated delay of a deliverable, as far in advance of the
due date as possible.

## Description of IV&V Contract Deliverables

+----------------+---------------------+-------------------------------+
| **[Task]       | **[Deliver          | **[Time Period]{.smallcaps}** |
| {.smallcaps}** | able]{.smallcaps}** |                               |
+----------------+---------------------+-------------------------------+
| 8.1.1 Develop  | IV&V Management     | To be initially delivered     |
| IV&V           | Plan                | within the first thirty (30)  |
| Management     |                     | days from the date of         |
| Plan           |                     | contract award, and updated   |
|                |                     | and delivered one-week prior  |
|                |                     | to the commencement of the    |
|                |                     | onsite portion of each        |
|                |                     | Initial and Periodic IV&V     |
|                |                     | Review. The IV&V Management   |
|                |                     | Plan shall contain the        |
|                |                     | following:                    |
|                |                     |                               |
|                |                     | a.  Resumes of all Key IV&V   |
|                |                     |     Service Provider          |
|                |                     |     personnel;                |
|                |                     |                               |
|                |                     | b.  A schedule describing the |
|                |                     |     next two-IV&V Review      |
|                |                     |     periods, including tasks, |
|                |                     |     activities, deliverables, |
|                |                     |     and milestones, and will  |
|                |                     |     show the schedule's       |
|                |                     |     critical path reflecting  |
|                |                     |     both IV&V Service         |
|                |                     |     Provider's and State's    |
|                |                     |     delivery and response     |
|                |                     |     milestones;               |
|                |                     |                               |
|                |                     | c.  An organization chart     |
|                |                     |     reflecting the IV&V       |
|                |                     |     Service Provider's team,  |
|                |                     |     including the team's      |
|                |                     |     place within the IV&V     |
|                |                     |     Service Provider's        |
|                |                     |     corporate structure, and  |
|                |                     |     providing the key names,  |
|                |                     |     addresses and other       |
|                |                     |     contact information to be |
|                |                     |     used for dispute          |
|                |                     |     resolution and customer   |
|                |                     |     feedback;                 |
|                |                     |                               |
|                |                     | d.  A narrative description   |
|                |                     |     of all deliverables,      |
|                |                     |     including expected        |
|                |                     |     format, content, and      |
|                |                     |     organization, to be       |
|                |                     |     developed and delivered   |
|                |                     |     during the next two IV&V  |
|                |                     |     Reviews (12 months); and, |
|                |                     |                               |
|                |                     | e.  As Appendices, all        |
|                |                     |     applicable, Project       |
|                |                     |     lifecycle-appropriate     |
|                |                     |     IV&V Checklists to be     |
|                |                     |     utilized during the next  |
|                |                     |     two IV&V Reviews.         |
+----------------+---------------------+-------------------------------+
| 8.1.2 Develop  | IV&V Checklists     | These are IV&V Checklists,    |
| IV&V Review    |                     | presenting in Question and    |
| Checklists     |                     | Answer format, elements to be |
|                |                     | reviewed, observed,           |
|                |                     | monitored, and commented on,  |
|                |                     | with regard to all aspects of |
|                |                     | industry standards for        |
|                |                     | Project Management, Software  |
|                |                     | and Systems Development, and  |
|                |                     | Engineering disciplines as    |
|                |                     | found in IEEE, CMI, and PMBOK |
|                |                     | industry standards, at a      |
|                |                     | minimum.                      |
|                |                     |                               |
|                |                     | The IV&V Checklists are to be |
|                |                     | compiled and delivered on an  |
|                |                     | ongoing basis, with the first |
|                |                     | checklists being delivered    |
|                |                     | applicable to the project     |
|                |                     | lifecycle phase to be         |
|                |                     | monitored and reviewed within |
|                |                     | the Initial IV&V Review       |
|                |                     | period, with such checklist   |
|                |                     | delivery made prior to the    |
|                |                     | onsite portion of the review  |
|                |                     | being performed. As IV&V work |
|                |                     | progresses and project        |
|                |                     | lifecycle phases change,      |
|                |                     | applicable, updated IV&V      |
|                |                     | Checklists will be delivered, |
|                |                     | as needed, prior to           |
|                |                     | commencement of the on-site   |
|                |                     | portion of that respective,   |
|                |                     | periodic IV&V Review.         |
+----------------+---------------------+-------------------------------+
| 8.1.3 Conduct  | IV&V Review         | The Initial IV&V Review will  |
| Initial IV&V   | Activities          | commence within sixty (60)    |
| Review         |                     | days from the date of         |
|                |                     | contract award, with the      |
|                |                     | first activity of the Initial |
|                |                     | IV&V Review being the onsite  |
|                |                     | review. The IV&V Service      |
|                |                     | Provider will be restricted   |
|                |                     | to conducting its onsite      |
|                |                     | review within a 10 calendar   |
|                |                     | day period. This onsite       |
|                |                     | portion of the Initial IV&V   |
|                |                     | Review will include the       |
|                |                     | following activities:         |
|                |                     |                               |
|                |                     | a.  Submit to IV&V Contract   |
|                |                     |     Manager a schedule of     |
|                |                     |     onsite review activities  |
|                |                     |     to be performed with      |
|                |                     |     State Project and         |
|                |                     |     Department;               |
|                |                     |                               |
|                |                     | b.  Submit to IV&V Contract   |
|                |                     |     Manager a list of Project |
|                |                     |     Team and Stakeholder      |
|                |                     |     interviews to be          |
|                |                     |     performed, documentation  |
|                |                     |     required to review; and;  |
|                |                     |                               |
|                |                     | c.  Submit to IV&V Contract   |
|                |                     |     Manager a list of Project |
|                |                     |     Documentation to be       |
|                |                     |     provided for IV&V Service |
|                |                     |     Provider review, and      |
|                |                     |                               |
|                |                     | d.  Submit to IV&V Contract   |
|                |                     |     Manager a list of Project |
|                |                     |     Meetings, etc., to be     |
|                |                     |     attended and observed by  |
|                |                     |     the IV&V Service          |
|                |                     |     Provider.                 |
|                |                     |                               |
|                |                     | Upon completion of the onsite |
|                |                     | portion of the review, the    |
|                |                     | IV&V Service Provider will    |
|                |                     | leave the Project site and at |
|                |                     | their own place of business   |
|                |                     | review and analyze collected  |
|                |                     | Project artifacts and draft   |
|                |                     | the Initial IV&V Review       |
|                |                     | Report.                       |
+----------------+---------------------+-------------------------------+
| 8.1.4 Initial  | Initial IV&V Review | a.  An Initial IV&V Review    |
| IV&V Review    | Report (Draft and   |     Report (Draft Version)    |
| Report         | Final)              |     will be delivered to      |
|                |                     |     Federal OCSE and IV&V     |
|                |                     |     Contract Manager (at same |
|                |                     |     time) sixty (60) calendar |
|                |                     |     days after the start of   |
|                |                     |     the onsite portion of the |
|                |                     |     Initial IV&V Review.      |
|                |                     |                               |
|                |                     | b.  Federal OCSE will review  |
|                |                     |     this Draft Version and    |
|                |                     |     provide comments and ACF  |
|                |                     |     Priorities that will be   |
|                |                     |     incorporated to the Draft |
|                |                     |     Version of the Initial    |
|                |                     |     IV&V Review Report and a  |
|                |                     |     revised report will be    |
|                |                     |     released to the State's   |
|                |                     |     Project and Department    |
|                |                     |     five (5) calendar days    |
|                |                     |     after receipt of ACF      |
|                |                     |     comments and Priorities   |
|                |                     |     to the Draft Version of   |
|                |                     |     the Initial IV&V Review   |
|                |                     |     Report.                   |
|                |                     |                               |
|                |                     | c.  State Project and         |
|                |                     |     Department comments to    |
|                |                     |     the Draft Version of the  |
|                |                     |     Initial IV&V Review       |
|                |                     |     Report will be returned   |
|                |                     |     to the IV&V Service       |
|                |                     |     Provider within 20        |
|                |                     |     calendar days of receipt  |
|                |                     |     of the Draft Version of   |
|                |                     |     the Initial IV&V Review   |
|                |                     |     Report.                   |
|                |                     |                               |
|                |                     | d.  The IV&V Service Provider |
|                |                     |     will correct mistakes of  |
|                |                     |     fact to the Draft Version |
|                |                     |     of the Initial IV&V       |
|                |                     |     Review Report, and append |
|                |                     |     to the Draft Version all  |
|                |                     |     other Department          |
|                |                     |     comments, and redeliver   |
|                |                     |     the Initial IV&V Review   |
|                |                     |     Report, marked as Final,  |
|                |                     |     to OCSE, the IV&V         |
|                |                     |     Contract Manager, the     |
|                |                     |     State Project and the     |
|                |                     |     Department. This Final    |
|                |                     |     Version of the Initial    |
|                |                     |     IV&V Review Report        |
|                |                     |     deliverable concludes the |
|                |                     |     Initial IV&V Review.      |
+----------------+---------------------+-------------------------------+
| 8.1.5 Conduct  | Periodic IV&V       | Periodic IV&V Reviews will    |
| Periodic       | Review Activities   | commence six (6) months       |
| Review(s)      |                     | following the start of the    |
|                |                     | previous IV&V review, with    |
|                |                     | the first activity of the     |
|                |                     | Periodic IV&V Review being    |
|                |                     | the onsite review. The IV&V   |
|                |                     | Service Provider will be      |
|                |                     | restricted to conducting its  |
|                |                     | onsite review within a 10     |
|                |                     | calendar day period. This     |
|                |                     | onsite portion of the Initial |
|                |                     | IV&V Review will include the  |
|                |                     | following activities:         |
|                |                     |                               |
|                |                     | a.  Submit to IV&V Contract   |
|                |                     |     Manager a schedule of     |
|                |                     |     onsite review activities  |
|                |                     |     to be performed with      |
|                |                     |     State Project and         |
|                |                     |     Department;               |
|                |                     |                               |
|                |                     | b.  Submit to IV&V Contract   |
|                |                     |     Manager a list of Project |
|                |                     |     Team and Stakeholder      |
|                |                     |     interviews to be          |
|                |                     |     performed, documentation  |
|                |                     |     required to review;       |
|                |                     |                               |
|                |                     | c.  Submit to IV&V Contract   |
|                |                     |     Manager a list of Project |
|                |                     |     Documentation to be       |
|                |                     |     provided for IV&V Service |
|                |                     |     Provider review, and,     |
|                |                     |                               |
|                |                     | d.  Submit to IV&V Contract   |
|                |                     |     Manager a list of Project |
|                |                     |     Meetings, etc., to be     |
|                |                     |     attended and observed by  |
|                |                     |     the IV&V Service          |
|                |                     |     Provider.                 |
|                |                     |                               |
|                |                     | Upon completion of the onsite |
|                |                     | portion of the Periodic       |
|                |                     | Review, the IV&V Service      |
|                |                     | Provider will leave the       |
|                |                     | Project site and at their own |
|                |                     | place of business review and  |
|                |                     | analyze collected Project     |
|                |                     | artifacts and draft the       |
|                |                     | respective Periodic IV&V      |
|                |                     | Review Report.                |
+----------------+---------------------+-------------------------------+
| 8.1.6 Periodic | Periodic IV&V       | a.  A Periodic IV&V Review    |
| IV&V Review    | Review Report       |     Report (Draft Version)    |
| Report         | (Draft and Final)   |     will be delivered to      |
|                |                     |     Federal OCSE and IV&V     |
|                |                     |     Contract Manager (at same |
|                |                     |     time) sixty (60) calendar |
|                |                     |     days after the start of   |
|                |                     |     the onsite portion of the |
|                |                     |     respective Periodic IV&V  |
|                |                     |     Review.                   |
|                |                     |                               |
|                |                     | b.  Federal OCSE will review  |
|                |                     |     this Draft Version and    |
|                |                     |     provide comments and ACF  |
|                |                     |     Priorities that will be   |
|                |                     |     incorporated to the Draft |
|                |                     |     Version of the Periodic   |
|                |                     |     IV&V Review Report and a  |
|                |                     |     revised report will be    |
|                |                     |     released to the State's   |
|                |                     |     Project and Department    |
|                |                     |     five (5) calendar days    |
|                |                     |     after receipt of ACF      |
|                |                     |     comments and Priorities   |
|                |                     |     to the Draft Version of   |
|                |                     |     the respective Periodic   |
|                |                     |     IV&V Review Report.       |
|                |                     |                               |
|                |                     | c.  State Project and         |
|                |                     |     Department comments to    |
|                |                     |     the Draft Version of the  |
|                |                     |     Periodic IV&V Review      |
|                |                     |     Report will be returned   |
|                |                     |     to the IV&V Service       |
|                |                     |     Provider within 20        |
|                |                     |     calendar days of receipt  |
|                |                     |     of the Draft Version of   |
|                |                     |     the respective Periodic   |
|                |                     |     IV&V Review Report.       |
|                |                     |                               |
|                |                     | d.  The IV&V Service Provider |
|                |                     |     will correct mistakes of  |
|                |                     |     fact to the Draft Version |
|                |                     |     of the respective         |
|                |                     |     Periodic IV&V Review      |
|                |                     |     Report, and append to the |
|                |                     |     Draft Version all other   |
|                |                     |     Department comments, and  |
|                |                     |     redeliver the Periodic    |
|                |                     |     IV&V Review Report,       |
|                |                     |     marked as Final, to OCSE, |
|                |                     |     the IV&V Contract         |
|                |                     |     Manager, the State        |
|                |                     |     Project and the           |
|                |                     |     Department. This Final    |
|                |                     |     Version of the respective |
|                |                     |     Periodic IV&V Review      |
|                |                     |     Report deliverable        |
|                |                     |     concludes the respective  |
|                |                     |     Periodic IV&V Review.     |
+----------------+---------------------+-------------------------------+
| 8.1.7 Prepare  | Formal debriefing   | If desired by and requested   |
| debriefing and | presentation(s) to  | by the Project Team, Agency,  |
| deliver for    | the Department,     | and Department, the IV&V      |
| OCSE           | Agency, Project,    | Service Provider will prepare |
|                | and OCSE on the     | and deliver a debriefing      |
|                | respective IV&V     | related to the latest,        |
|                | Review Report.      | respective (Initial or        |
|                |                     | Periodic) IV&V Review         |
|                |                     | Report's results to the       |
|                |                     | Department, Agency, Project,  |
|                |                     | and OCSE. Any such debriefing |
|                |                     | must be conducted within 5    |
|                |                     | calendar days of delivery of  |
|                |                     | the Final Version of the      |
|                |                     | respective (Initial or        |
|                |                     | Periodic) IV&V Review Report. |
|                |                     | Debriefings prior to this     |
|                |                     | milestone within the IV&V     |
|                |                     | Services contract, whether    |
|                |                     | during the course of an       |
|                |                     | onsite review, or subsequent  |
|                |                     | IV&V Service Provider review, |
|                |                     | analysis, and report creation |
|                |                     | timeframe, or prior to        |
|                |                     | delivery of the respective    |
|                |                     | IV&V Review Report under this |
|                |                     | contract, is prohibited.      |
+----------------+---------------------+-------------------------------+
| 8.1.8 Prepare  | Contract invoicing. | No more than once a month     |
| and deliver    |                     | during active work conducting |
| invoices for   |                     | a semi-annual (Initial or     |
| payment.       |                     | Periodic) IV&V Review.        |
+----------------+---------------------+-------------------------------+
| 8.1.9 Prepare  | Contract Status     | No more than once a month     |
| and deliver    | Reporting To IV&V   | during active work conducting |
| Monthly Status | Contract Manager.   | a semi-annual (Initial or     |
| Reports.       |                     | Periodic) IV&V Review, inform |
|                |                     | the IV&V Contract Manager of  |
|                |                     | current contract status,      |
|                |                     | availability of IV&V Service  |
|                |                     | Provider key personnel, work  |
|                |                     | and deliverables expectations |
|                |                     | prospective to the next 60    |
|                |                     | days in contract schedule.    |
+----------------+---------------------+-------------------------------+
| 8.1.10         | Performed reviews   | If desired and requested by   |
| Deliverable    | of project          | the Project Team, Agency, and |
| Observation    | artifacts,          | Department, the IV&V Service  |
| Report (DOR)   | processes or        | Provider will prepare and     |
|                | deliverables not    | deliver a one-time, focused,  |
|                | otherwise defined   | specific Deliverable          |
|                | herein this scope   | Observation Report to the     |
|                | of work, as-needed  | IV&V Contract Manager (for    |
|                | in a special scope  | delivery to the State         |
|                | of work between     | Project, etc.,) and OCSE, at  |
|                | State and IV&V      | the same time, presenting an  |
|                | Service Provider.   | analysis of a prescribed      |
|                |                     | deliverable or other task not |
|                |                     | specifically referenced by    |
|                |                     | this scope of work. Examples  |
|                |                     | of such focused Deliverables  |
|                |                     | Observation Reports include:  |
|                |                     | a network capacity,           |
|                |                     | bandwidth, and throughput     |
|                |                     | analysis; and, independent    |
|                |                     | analysis of compliance of a   |
|                |                     | project deliverable with      |
|                |                     | contract specifications, etc. |
|                |                     | The State Project, Agency,    |
|                |                     | and Department may receive a  |
|                |                     | debriefing on the results of  |
|                |                     | such a DOR from the IV&V      |
|                |                     | Service Provider only with    |
|                |                     | the concurrence and           |
|                |                     | attendance of OCSE.           |
+----------------+---------------------+-------------------------------+
| 8.1.11 Archive | Periodic Archive    | A complete CD-ROM archive of  |
| Documents      | Creation and        | all IV&V Documents including  |
|                | Delivery of all     | draft and final reports,      |
|                | project artifacts   | status briefings, exception   |
|                | and research        | reports, all versions of the  |
|                | materials and       | Project Management Workplan   |
|                | contract            | (PMW). Deliverable            |
|                | deliverables        | Observation Review (DOR)      |
|                |                     | Reports, Monthly (Financial)  |
|                |                     | Invoicing, Project Status     |
|                |                     | Reports, and all project      |
|                |                     | materials, documentation,     |
|                |                     | artifacts, data, reports,     |
|                |                     | forms, etc., collected by the |
|                |                     | IV&V Service Provider during  |
|                |                     | the course of their latest    |
|                |                     | IV&V Review. This complete    |
|                |                     | archive is to be submitted    |
|                |                     | with the respective final     |
|                |                     | invoice for the IV&V Review   |
|                |                     | period in question.           |
+----------------+---------------------+-------------------------------+

All deliverables shall be approved by the State in order for the task
which produced them to be considered complete. In all cases, payments to
the IV&V provider shall be contingent upon State approval of
deliverables. No review will be considered complete until the approved
documentation is delivered to and reviewed by the cognizant Federal OCSE
and the State.\
\
For instances wherein the IV&V Service Provider delivers a one-time,
focused, whether solicited by the State or unsolicited, the proposal for
review of some project artifact, process, or deliverable, called a
Deliverable Observation Review (DOR), must include descriptions of the
actions that shall be taken to produce the DOR Report, a proposed format
and content outline for each DOR deliverable, and obtain State approval
prior to any commencement of work.\
\
The State must approve, in writing, changes to milestones, deliverables
or other material changes to the contract prior to implementation of
changes. The State may require concurrence of the Federal OCSE in any
changes prior to their implementation.

# IV&V Format and Content Reporting Requirements

All deliverables, reports, analyses, etc., whether in draft or final,
must be delivered by the IV&V Service Provider directly to the Federal
OCSE at the same time they go to the State. In this regard, in addition
to the Federal OCSE, the IV&V Service Provider should ensure delivery to
the IV&V Contract Manager who is the State entity responsible for IV&V
deliverables dissemination to the State's Project, Agency, Departmental,
and Stakeholder personnel. The State may not modify, or reject any IV&V
Review Report beyond recommendations to emend mistakes of fact. State
comments to all IV&V Review Report findings will only be appended to the
respective report.\
\
For each area evaluated, the report should contain the current status of
the State\'s effort, including any pertinent historical background
information. The report should also contain a detailed analysis of each
area, which answers, at the least, the following general questions:

-   What is the State\'s current process in this area?

-   What\'s good about the State\'s process?

-   What about the State\'s process or technology needs improvement?

-   Is the State making measurable progress in this area?

-   Is the effort within the triple constraints of budget, scope, and
    schedule?

-   What standards is the project following (State, industry \[IEEE,
    SEI, ISO, etc.,\] internally?

-   Is the appropriate documentation and other project artifacts
    accurate and up-to-date?

-   Is the adequate Stakeholder involvement in the Project?

-   Are best practices and metrics employed to identify issues,
    progress, performance, etc?

Responses should be quantified whenever possible. The report should also
contain detailed recommendations in each area specifying what can be
done immediately and in the long term to improve the State\'s operation.
Any technologies, methodologies, or resources recommended should reflect
industry standards and be appropriate for the unique circumstances and
constraints of the \[insert Project name\] Project. The recommendations
should also specify a method of measuring the State\'s progress against
the recommendations.

Follow-up reports should have quantified information on the progress
that the State has made against the recommendations from the previous
review. The follow-up report should also contain any additional and/or
modified recommendations at the same level of detail as the initial
recommendations. All report findings and recommendations should be
historically traceable (with a clear and consistent method of
identification/numbering) from the time they are first reported by the
IV&V Service Provider until closure.\
\
The deliverables for this contract shall be provided in hardcopy form
and on electronic media, using the following software standards (or
lower convertible versions):

  ----------------------------- -----------------------------------------
  **[Document                   **[Format]{.smallcaps}**
  Type]{.smallcaps}**           

  Word Processing               Microsoft Word 2003, or newer

  Spreadsheets                  Microsoft Excel 2003, or newer

  Graphics                      Microsoft PowerPoint 2003, or newer

  Project Management            Microsoft Project 2003, or newer
  ----------------------------- -----------------------------------------

As previously stated, all drafts and final deliverables shall be
provided to the Federal OCSE at the same time they are provided to the
State (e.g., IV&V Contract Manager). As previously stated, the State
cannot modify or reject a report prior to submission.

# State Furnished Items

-   Workspace for up to three contractor staff while on-site at the
    project for the duration of the contract. The workspace will include
    desk or tables, phone and access to the projects' LAN. The
    contractor is expected to have regular office space separate from
    the \[insert Project name\] Project site.

-   Access to \[insert Project name\] Project information, including,
    but not limited to, technical documentation and \[insert Project
    name\] Project status data.

-   Access to State and contractor project personnel for information
    related to the project.

-   The State is [not]{.underline} responsible for providing clerical or
    administrative support to the IV&V Service Provider.

# Travel

IV&V providers will be required to travel to the State offices
periodically. All travel must be pre-approved by the IV&V Contract
Manager. Reimbursement of all Per Diem and Meals and Incidentals
Expenses (M&IE) costs for travel purposes, if needed, shall be based on
current, approved GSA-Travel Schedule rates. All Offerors to this
solicitation will include in their bid, a fixed amount, by year, for
travel under this contract. This standard contract travel budget will be
\$20,000 per year. Offerors are advised to bid exactly this amount.

# Points of Contact

The State points of contact for this SOW are:

  ----------------------- ------------ ------------ -----------------------
  Name                    Phone(s)     Fax          E-mail

                                                    

                                                    
  ----------------------- ------------ ------------ -----------------------

# Contract Period

It is anticipated that the start date of these services will be
\_\_\_\_\_\_\_\_\_\_ and the end date will be \_\_\_\_\_\_\_\_\_.
Contract execution is contingent on prior Federal approval of the
resulting IV&V contract, and upon the availability of state funding.
Offerors to this solicitation will be presenting proposals for a period
of performance of one 12-month Base Year Period followed by four
consecutive 12-month Option Year Periods, for a total period of
performance if all option periods are exercised of 60-months.

# Definition of Terms

All terms are as defined in the Federal Acquisition Regulation (FAR) 48
CFR Chapter 1 Part 2 - Definitions of Words and Terms, including
amendments effective as of Federal Acquisition Circular FAC 97-25, May
02, 2001.