***[CIO as IV&V Provider]{.underline}***

**Question 1A:** We would like to have our State\'s Chief Information
Officer (CIO) be the IV&V Overseer. Since the CIO is in a different
department, and reports directly to the Governor/Lt. Governor, it seemed
removed enough to satisfy the requirement. Is this acceptable to OCSE?
*\[February 2000\]*

**Answer 1A:** Without benefit of an actual organization chart,
capabilities statement, and description of duties on the part of the
CIO; the CIO would be considered acceptable to provide IV&V services
provided the following assumptions are true regarding the CIO position:
(1) Organizational independence from the State\'s IV-D program and its
umbrella agency, the State\'s Department of Social Services (DSS); and
(2) the Office of the CIO has the technical resources necessary to
perform the work of the IV&V Service Provider, either through in-house
staff or through contract resources.

**Question 1B:** Does the CIO govern/oversee/manage the selection
process? We presume the CIO participates, to what level is the question
at hand. *\[February 2000\]*

**Answer 1B:** The Office of the CIO is the exclusive agency responsible
for performance of all IV&V activities. This includes all governance,
oversight and management of IV&V planning, the conduct of IV&V reviews,
and the generation of reports of findings and recommendations of those
IV&V reviews. IV-D agency and the State\'s DSS participation shall be
limited to support for the IV&V reviews conducted by the CIO, relative
to providing for interviewing of cognizant staff, and providing for the
availability of necessary project and system documentation, software,
code, plans, etc., as deemed appropriate and necessary by the CIO in the
execution of the IV&V reviews.

**Question 1C:** Does HHS (ACF/OCSE) have a part in the selection
process? We presume yes, but not in charge and perhaps not a voting
member? *\[February 2000\]*

**Answer 1C:** HHS (ACF/OCSE) **does not** have a part to play in the
selection process. Procurement processes are the sole dominion of
applicable State procurement laws and regulations. However, the
procurement process \[Request for Proposal (RFP), Invitation to Bid
(ITB), etc.,\] and subsequent contract are subject to the **prior
approval** provisions of Federal regulations at 45 CFR Part 95 and 45
CFR Part 307.15(b)(10). Our Office stands ready to provide technical
assistance to your Office as you pursue the procurement process.

**Question 1D:** Does ACF/OCSE participate in the selection process
(Regional Office, Central Office or someone)? *\[February 2000\]*

**Answer 1D:** Neither ACF, OCSE Central Office or Regional Offices
participate in the procurement process. This exclusion includes the
creation of the procurement vehicle (RFP, IFB, etc.), proposal
evaluation process (including evaluative criteria), and actual selection
of the successful offeror to the solicitation for an IV&V Service
Provider. We would note, however, that upon creation of the procurement
vehicle (e.g., RFP) and prior to its release for bids, it must first be
submitted to OCSE Central Office for **review and prior written
approval**. Likewise, upon completion by the State of the selection of
an offeror to the solicitation, and prior to actual execution of a
contract with the successful offeror, OCSE Central Office must again
**review and provide prior written approval** of the contract. We would
also note that funding for the requisite IV&V Service Provider\'s
contract will be provided through the Advance Planning Document (APD)
process. Federal written approval of the RFP and resultant contract are
distinct from Federal funding approval for the same. Funding will be
provided through the APD for the project.

**Question 1E:** Can HHS (ACF/OCSE) pay the IV&V vendor bills, if they
are signed off by the CIO as the overseer? The State would set up an
Interagency Cooperative Agreement so they didn\'t have to get an
appropriation just for a pass through (seems redundant). We presume this
would be allowed. *\[February 2000\]*

**Answer 1E:** Expenditures for the IV&V Service Provider are eligible
for Federal financial participation (FFP) at the regular (66 percent)
rate of reimbursement. You are correct in your assumption that payment
for the expenditures of an IV&V Service Provider could be made through
an Interagency Agreement with the CIO. The Interagency Agreement will
necessarily need to incorporate certain performance criteria and
conditions, as delineated in ACF/OCSE\'s IV&V Assessment Review Report
for your project. Specifically, these criteria are:

-   The IV&V Service Provider (CIO) must supply all plans, reports,
    analyses and recommendations to OCSE Central Office **at the same
    time** it is provided to the Child Support Enforcement Unit of the
    State's Department of Social Services, as specified in 45 CFR
    307.15(b)(10)(ii).

-   The State must submit an Advance Planning Document Update (APDU)
    showing all IV&V activities and their costs (which are eligible for
    Federal financial participation at the regular \[66 percent\]
    matching rate.)

-   Procurement of an IV&V Service Provider (if conducted by the CIO),
    including the resultant contract and the interagency cooperative
    agreement are subject to **prior approval** by OCSE Central Office
    as specified in Action Transmittal OCSE-AT-98-26.

-   In addition to the IV&V effort\'s scope of work, deliverables, and
    schedules, the solicitation document and resultant contract and
    interagency cooperative agreement (with/between the DSS and CIO)
    must include the names and qualifications of key personnel (CIO
    staff and if used, contractor staff) who will actually perform and
    manage the IV&V services.

**Question 1F:** Could the CIO hire help with the oversight of the IV&V
process if it was needed, and then with an interagency cooperative
agreement, have HHS (ACF/OCSE) pay the cost? We presume this would be
allowed. *\[February 2000\]*

**Answer 1F:** Yes. Please refer to the above response for specific
explanations and conditions.

***[Metrics vs. Matrix]{.underline}***

**Question 2:** In the Recommendations Section of our State\'s CSES
project IV&V Assessment Review Report it states: \"Develop performance
metrics, which allow tracking of project completion against milestones
set by the State. Monitor the performance of the Quality Assurance (QA)
function/organization by reviewing its reports and performing spot
checks of system documentation.\" Is the word \"metrics\" a typo or was
some other word intended such as matrix? *\[March 2000\]*

**Answer 2:** \"Metrics\" is what was intended. Metrics can be any
measurable data taken during the program/software development cycle.
Software size and percentage complete are [examples]{.underline} of
metrics. Typical software size metrics are lines of code (LOC), function
points and units (also called procedures, subroutines, subprograms or
modules). Whichever measure is used should be appropriate for your
system\'s computer language(s) and design methodology and should be
applied consistently across the project.

Other examples of metrics include number of software requirements,
number of changes made to the software, number of defects found, a count
of where in the development process defects are found, number of
outstanding defects to be fixed and their criticality, number of defects
found per test or per test phase, etc, etc, etc.

***[IV&V Contract Management]{.underline}***

**Question 3A:** Per discussion with OCSE, we are exploring the idea of
having the IV&V contract managed by the State\'s Department of Revenue
(DOR) Inspector General (IG) office. We have presented the idea to the
IG and if the DOR agrees, then it will be presented to the State\'s IV-D
CSES Board.

It was mentioned that OCSE needs proof of the IG\'s independence from
IV-D. Would an organizational chart showing that the IG reports directly
to the Executive Director suffice? Does OCSE need anything else?
*\[March 2000\]*

**Answer 3A:** An organizational chart and a \"mission\" statement
(i.e., what are the organizational responsibilities, to whom does each
organization report, etc.) should contain information on their
operational independence.

**Question 3B:** If we choose the above option, the IG would contract
with a contractor from the State contract list, using a Statement of
Work (SOW) based on the approved SOW, but modified to show the DOR
Inspector General wherever contract responsibilities are mentioned. Will
OCSE need to prior approve the revised document and the contractor\'s
proposal (which will not change except for date references)? If so, we
could send OCSE a redlined version with all changes highlighted to make
the OCSE review simpler. *\[March 2000\]*

**Answer 3B:** **Yes.** OCSE will need the new contract between the IG
and the selected IV&V contractor. Yes, a redlined version of changes
would be helpful.

**Question 3C:** Since the IG is in our same agency (DOR), do we still
need a Memorandum of Understanding (MOU) between them and us? *\[March
2000\]*

**Answer 3C:** **Yes.** The IG will need a MOU with IV-D outlining the
IG\'s responsibilities regarding handling of the IV&V efforts on CAMS,
as well as that the IG will not be \"editing for public consumption\"
drafts of IV&V reports, deliverables, etc. but will forward and direct
the IV&V contractor to forward any drafts to OCSE when they are
delivered by the contractor to the IG and then by the IG to IV-D.

[**Question 3D:** The regulations seem to suggest that the agency
issuing the IV&V Request for Proposal (RFP) and/or initiating the IV&V
contract (in our case, the Department of Social Services) could also be
the agency to which the IV&V vendor is responsible regarding
performance, e.g. for the purpose of payment under the contract. Could
you provide clarification on this issue? *\[January 2001\]*]{.mark}

[]{.mark}

[**Answer 3D:**. You correctly have interpreted the regulatory language
you cite as potentially allowing the State's Department of Social
Services (DSS) to operationally manage the IV&V contract. However, our
Office has determined that there is a real and perceived conflict of
interest in the State's DSS managing the IV&V Service Provider contract,
and thus will require the contract to be managed by a separate State
agency. This agency could be a State Department of Information
Technology, State Auditor's Office, or State Department of the Treasury,
as examples. For the purpose of payment and overall scope of work,
management responsibilities, etc., we look for a "Memorandum of
Understanding" or similar interagency agreement between the DSS and
whatever State agency is contracted with to provide the State's Child
Support Enforcement System (CSES) project with IV&V services.]{.mark}

***[Prior Approval / Approval]{.underline}***

**Question 4A:** Is it necessary to have the IV&V Request for Proposal
(RFP) approved by OCSE Central Office (CO) **before** publishing it to
the public? If so, how much time should be allowed for approval by CO
after it is submitted? *\[March 2000\]*

**Answer 4A:** **Yes.** The RFP and any resulting contract **require the
prior written approval** of OCSE CO **prior to their release or
execution**, respectively. The State should allow up to 60 days for
Federal response to a written request for prior approval. It is the
intent of this office to expedite these requests, given the need for
such IV&V services.

**Question 4B:** Does the State need to receive approval of the
recommended IV&V Service Provider before the Proposal Evaluation
Committee issues its Notice of Intent to Award Contract? If approval is
required, how much time should be allowed for approval from OCSE Central
Office before we issue the Notice of Intent to Award? *\[March 2000\]*

**Answer 4B:** **No.** The selection process regarding vendor selection
for IV&V service is a State-specific process that does not incorporate
Federal intervention. However, please note that the procurement must at
a minimum, meet the procurement requirements found in Federal
regulations at 45 CFR Part 74.40 through 74.62 in order for the
procurement to receive Federal approval and thus, Federal funding.
Regarding the length of time to allow for Federal review and approval,
the 60-day timeframe applies.

***[[IV&V Preferred Providers List]{.underline}]{.mark}***

[]{.mark}

[**Question 5:** Does the Administration for Children and Families (ACF)
have a "preferred list" of vendors who perform IV&V services and would
ACF provide that list to the State? If not, could ACF provide a list of
vendors that have performed this required IV&V service well in another
State? *\[January 2001\]*]{.mark}

[]{.mark}

[**Answer 5:** ACF **does not** have a "Preferred Providers Listing". We
have, for the purposes of technical assistance, gathered a list of
vendors from various GSA sites that do provide, to varying degrees, the
types of services that constitute IV&V. Again, we must make clear that
ACF does not and cannot endorse, promote, or otherwise attest to the
quality or professionalism of any of the firms listed relative to IV&V
services. This data is gathered from the U.S. General Services
Administration.]{.mark}

***[[Change in Scope of IV&V]{.underline}]{.mark}***

[]{.mark}

[**Question 6:** In the Executive Summary in OCSE's most recent IV&V
report it states, "This report sets forth requirements for the State to
initially provide for a continuous, ongoing IV&V presence. This is a
change from OCSE's initial IV&V Assessment Review report, which required
IV&V reviews every 6 months." To better understand what level of
participation and resources the State would require of the entity
providing the IV&V services, could you better define "continuous,
ongoing presence"? *\[January 2001\]*]{.mark}

[]{.mark}

[**Answer 6:** Our most recent IV&V review found that the need for IV&V
services, based on the system development plan proposed by the State's
Feasibility Study necessarily has increased from our original assessment
of once every 6 months, to a continuous, ongoing presence. This ongoing
presence defines a full-time staff of most likely two to four IV&V staff
(depending on their skills and abilities and scope of work). This
staffing could increase, on an as-needed basis by the State's project,
to conduct "go/no go" assessments of the project for senior management
at pre-defined, critical project milestones, such as completion of the
Detailed Design documentation, completion of a phase-one application
coding, or prior to commencement of pilot test. Such increased staffing
could also be used for one-time activities such as a system-wide stress
test or capacity analysis.]{.mark}