1)  With regard to the agencies with which the new common collection
    system is required to interface and communicate:

    a.  Do they currently have capabilities to accept externally
        provided data from 'foreign' systems?

### Answer: Currently, that capacity does not exist in all systems.

b.  Is this capability real-time, accessible via the public Internet?

### Answer: See above.

c.  Who will construct, install and QA these interfaces -- the
    contractor or the agencies?

### Answer: The contractor.

d.  Will the agencies make available suitable QA environments for
    validation

> of the interfaces developed under this RFP?

###### Answer: Yes, the agencies will make available suitable environments as 

###### requested by the contractor through a formal request made through the agencies 

###### coordinators following award of the contract.

2)  Will the proposed Phase I system replace the existing collection
    channels into the four target systems after appropriate validation,
    or will there be an on-going requirement for the new Phase I common
    system to operate in parallel with existing legacy data collection
    channels?

## Answer: This will be a replacement system. However, there will be a need for parallel operations during a pre-production testing phase. 

3)  The RFP references specific CDC & FDA systems that already gather
    and collate various data. Do AHRQ and CMS who are also referenced in
    the RFP have other, unspecified data gathering systems with which
    the new system will also have to interface?

**Answer: AHRQ and CMS have systems that are to be integrated as part
of**

**Phase II of the overall integration plan. Phase I, which encompasses
integration,**

**does not require integration of CMS and AHRQ systems**.

4)  The RFP volume estimate only references a statistical calculation
    based on the number of incidents anticipated per patient bed-month.
    Is there any requirement or volume contribution that is anticipated
    to be generated by ambulatory and/or outpatient and satellite clinic
    operations?

Answer: Ambulatory and outpatient facilities are potential users of the
system

along with inpatient facilities. There are currently no estimates of the
number of

reports possible for these facilities.

5)  Privacy and non-traceability/non-accountability of reports is
    specifically mentioned in the RFP as a vehicle to encourage full
    reporting; this raises various questions:

    a.  How untraceable must the submission be; to the submitter; to the
        institution; to the terminal used to submit?

    b.  Given the data is encrypted from source and into the database,
        is there any requirement for an escrow capability to trace back
        to the origin?

    c.  If there are any conflicts with other national policies on
        traceability of such reports, how will these be resolved and
        will the contractor be inured against any adverse consequences?

Answer: These issues will be addressed by the PSTF coordinating
committee

following contract award.

6)  The authentication suggested for reports submitted is to use a PKI
    certificate approach. Without the use of some kind of
    challenge/response system and the use of a physical token, the use
    of certificates is no stronger than using a long password. Can
    another approach be utilized?

## Answer: An alternative approach can be proposed. However, any alternative 

## must take into consideration the authentication in use by the underlying 

## databases and systems. This is an issue for coordination and negotiation by the 

## Government's technical personnel in conjunction with the contractor after 

## contract award.

7)  It is not clear what authentication requirements are for those
    permitted to make inquiries of the data warehouse. Are there any
    authentication requirements, or will the system be available for
    essentially public use?

Answer: Those who submit reports will be able to initiate individual
queries of

the system.

8)  Can the proposed central system rely upon specific hardware,
    software and database products or does it have to be constructed in
    a product-agnostic fashion?

**Answer: It would be best to develop the system in as \"open\" a manner
as**

**possible. It is the intent of the Government to provide the reporting
system in a**

**publicly available format (pp 17 -18 of the RFP). It is incumbent upon
the**

**offeror to address issues this may cause them in their response.**

9)  Can the proposed system make use of software that is available under
    a GPL license and hence is required to be made publicly available
    under the licensing rules for using existing GPL licensed software?

Answer: GPL (General Public License,

http://www.gnu.org/copyleft/gpl.html#SEC1) - this would mean that any

software derived/enhanced out of the initially developed software would
also be

subject to a GPL, making any subsequent software also subject to the
GPL,

meaning it has to be freely distributed also. Since anything developed
under this

contract is owned by the Government, it is by definition, available for
public use.

So the answer would be yes.

10) If components of the system are proprietary, will those continue to
    be protected upon incorporation into the product?

## 

## Answer: Open architecture is preferred with the avoidance of proprietary 

## software.

11) For testing and acceptance of the system delivered under this Phase
    I RFP, will this be limited to a centrally implemented and installed
    system, or will it have to encompass the proposed intra-institution
    implementations that are referenced in the RFP?

Answer: Phase I involves testing of the common web-based user input and

prototype data warehouse.

12) May two small businesses co-propose and conduct project jointly?

**Answer: In accordance with FAR 9.601, two or more companies may form
a**

**partnership or joint venture to act as a potential prime contractor,
however, this**

**must be done formally and legally. The Small Business Administration
should**

**be able to provide assistance.**

13) May a large business be a subcontractor and conduct more than 50% of
    the work if there is a justification for it?

**Answer: No. In a Total Small Business Set-Aside, at least 50 percent
of the cost**

**of contract performance incurred for personnel shall be expended by
employees**

**of the small business.**

14) Is the vendor responsible for recruiting the pilot project
    hospitals?

## 

## Answer: Yes

15) Is 50 hospitals the target or the minimum requirement for the pilot
    project? If so, what kind of assistance will be provided by AHRQ?

**Answer: Yes 50 is a minimum target. AHRQ and other agencies of the
PSTF**

**will assist in identifying and recruiting candidate institutions for
the pilot**

**project.**

16) In the synopsis published online 6/18/02 there is a note indicating
    that only very small businesses of less that 15 employees and \$1
    million in revenues could propose. There is also a note requiring
    proposers to be minority institutions. Are these requirements
    correct?

**Answer: This note was not inserted by AHRQ and is not correct. The
size**

**standard is \$18 million in annual receipts to qualify as a small
business under**

**this requirement.**

17) Will phase II and III be automatically awarded to the successful
    contractor upon completion of phase I?

Answer: No. It will depend on the circumstances at the time.

18) Clarify the specific definition of small business (e.g., what is the
    revenue figure and as that an average over a number of years or
    current year revenue)

Answer: For purposes of this requirement only, to qualify as a small
business,

concerns must have annual receipts of less than \$18 million. FAR Part
19 states

"Annual receipts of a concern which has been in business for 3 or more
complete

fiscal years means the annual average gross revenue of the concern taken
for the

last 3 fiscal years." See FAR 19.1 for further details.

19) Can we receive the questions submitted by other bidders and the
    question answers provided by AHRQ?

Answer: This amendment includes all of the questions by other bidders
and

answers by AHRQ.

20) Will the contractor selected for Phase 1 have any conflict of
    interest in bidding Phase II and Phase III?

Answer: We do not anticipate at this time that there would be a conflict
of

interest.

21) Page 22, Item 2.11 appears to have several typographical errors,
    which make it impossible to determine the specific requirement.
    Please restate this paragraph.

**Answer:**

11. **The contractor shall submit 5 copies of a detailed implementation
    plan for Phase II, data integration and design of the data
    warehouse. A Phase III implementation plan for the potential
    inclusion of other reporting systems outside DHHS, including but not
    limited to VA, DoD, NRC, JCAHO, and State systems, must also be
    submitted to the Project Officer and Co-Project Officers for
    approval.**

```{=html}
<!-- -->
```
22) Page 26, Section F.3 Delivery Schedule, Item 6 calls for a draft of
    training material kit to be submitted 12 months from EDOC. Since
    this comes three months before even the initial demonstration of the
    prototype system, we assume that by "draft" the government means a
    plan with a detailed outline and general explanation of what the
    training kit will contain, rather than an actual draft training
    document. Is this assumption correct?

## Answer: Yes

23) Page 26, Section F.3 Delivery Schedule, Item 6. Would the government
    provide additional explanation of what is meant by "user based
    software"? Is a draft of software to be provided?

Answer: User-based software is intended to be a set of software for
local

institutions to be used in setting up their own internal event reporting
system

that is compatible with the Federal system. Thus, it would mirror the
front end

reporting as well as the structure of the prototype warehouse. While
local users

(hospitals and other healthcare institutions) would not be required to
use the

software, having software available will encourage reporting. The
contractor is

responsible for making the software available to pilot project
institutions.

24) Page 73, Management Plan, Item (2) requires a person-loading chart.
    Page 74, Qualifications of Proposed Staff, Including Consultants
    also requires a person-loading chart. Please clarify.

Answer: An integrated person-loading chart is desired for all work of
both the

prime contractor and subcontractors. Separate charts could be used for

clarification.

25) Page 74, Qualifications of Proposed Staff, Including Consultants
    asks for resumes of key personnel. Does the government require a
    certain minimum number of key personnel, or that certain project
    labor categories or project roles be designated as key personnel?

## 

## Answer: No

26) Page 74, Qualifications of Proposed Staff, Including Consultants
    contains extremely restrictive experience requirements for the
    Project Director, as well as for "approximately one-half of the
    proposal staff..." In particular, the requirements that staff have
    extraordinary depth in both health-related fields and
    technology-related fields make it difficult for any more than a
    handful of people to qualify for this contract. Given that this is a
    small business set-aside acquisition, we believe this will narrow
    the potential offerors to perhaps one or two companies, thereby
    limiting competition, which is not in the government's interest --
    either from a technical solution perspective or a cost perspective.
    In addition, the personnel restrictions will force small businesses
    to bring large companies on to their team, merely to qualify, which
    will tend to undermine the reasons for making this a small business
    set-aside. Will the government ease the personnel qualification
    restrictions and allow small businesses to determine what personnel
    are best suited to perform this work?

Answer: The technical proposal instructions under Section L.9 of the RFP
with

respect to Organizational/Corporate Experience and Qualifications of
Proposed

Staff, Including Consultants shall be revised to read as follows:

**3. Organizational/Corporate Experience**

> Offerors should list and summarize any **relevant** contracts (state
> or federal) or grants (state, federal, or private foundation) recently
> completed (within the last 3 1/2 years - since January 1, 1999), or
> that are currently in process, and describe the relevance to the
> tasks, sub-tasks, and associated activities that may be performed
> under this contract. The Offeror shall demonstrate the extent,
> relevance, and quality of their corporate experience as it relates to
> the requirements of this acquisition, including the following:
>
> At least 5 years of experience and demonstrated success in:
>
> Developing web-based user input systems for **large data systems**;
>
> Developing formal systems for monitoring and maintaining efficiency
> and quality in the use of computer and programmer resources;
>
> Development of training materials and support resources for computer
> database entry and analysis systems
>
> Dealing with **large complex data systems** and databases;
>
> Demonstrating the ability to maintain and manage multiple complex
> activities concurrently at the highest level of professional and
> scientific quality.

At least 3 years of experience in:

> **Patient safety**
>
> **Event reporting and error identification**
>
> **Healthcare data systems**

**4. Qualifications of Proposed Staff, Including Consultants**

> The offeror shall provide (1) the resumes of all key personnel
> (generally senior and junior technical staff) describing their
> qualifications as they relate to the requirements of this solicitation
> and (2) a person loading chart. The offeror is expected to be specific
> in describing the proposed personnel and their relevant qualifications
> and experience, including their background and experience as they
> relate to the requirements of this acquisition.
>
> The offeror should also describe:
>
> 1\) The experience of the Project Director as it relates to the
> requirements of this acquisition as evidenced by educational
> attainment, employment history, experience and specific professional,
> scientific or technical accomplishments, including the minimum
> experience requirements below. The Project Director should be a highly
> qualified senior staff member who is available on a day-to-day basis
> to direct and monitor the project contract and the associated
> technical tasks.

At least 5 years of experience in each of the following:

> Directing the development and maintenance of **large data systems**;
>
> Data processing management, including responsibility for the
> recruitment and supervision of programming staff, directing multiple
> simultaneous data processing tasks, providing fiscal controls, and
> overseeing technical components in a timely and efficient manner;
>
> The Project Director must also have experience exhibiting:
>
> Excellent overall project management skills that include
> substantive/technical areas, teamwork, budget management, cost
> control, flexibility, and the ability to produce deliverables on‑time,
> within budget, and of exceptionally high technical quality;
>
> Excellent verbal and written communication skills.
>
> 2\) The experience of staff and consultants as it relates to the
> requirements of this acquisition as evidenced by educational
> attainment, employment history, experience and specific professional,
> scientific or technical accomplishments.
>
> Minimum requirements with respect to specific types of programming
> skill/experience are given below: Proposed staff should have the
> following
>
> skill mix, either individually or collectively:
>
> 5 years or more of experience regularly web based user input formats
>
> 2 years or more of formal education in a health‑related field or
> social science;
>
> 3 years or more of experience in experience in using other large
> databases such as, but not limited to, data from the, CDC, CMS, FDA
> and/or AHRQ data systems;

3 years or more experience in health care coding and analysis systems;

> 3 years experience in ICD coding and clinical software (DRGs, disease
> staging, etc.);
>
> At least one programmer with 2 or more years experience in HTML
> programming and other web site support activities.
>
> In addition, the contractor must provide:
>
> At least 2 **members** of the staff should be highly organized and
> detail oriented with excellent communication skills with 3 or more
> years of experience in coordinating with outside agencies, preparing
> agreements for the uses and restrictions of their data, overseeing the
> process of data purchase and collection, and providing technical
> assistance to data organizations.
>
> At least one **member** of the staff should have expertise with data
> confidentiality and security issues.
>
> At least one **member** of the staff should have expertise in the area
> of patient safety and medical error **reporting.**

27) Is MEDSTAT precluded from bidding on this contract as a prime
    contractor?

Answer: Yes. They are a large business.

28) Is MEDSTAT precluded from bidding on this contract as a
    subcontractor, as part of a team?

###### Answer: Yes, in accordance with FAR Part 9.5 Organizational Conflicts of 

**Interest.**

29) As referenced in the FBO Synopsis description it stipulates that,
    \"The PSTF (Patient Safety Task Force) contracted with the MEDSTAT
    Group, inc. for an implementation planning study which recommended a
    phased integration process with three (3) distinct phases.\" It is
    apparent from the RFP Section C that the MEDSTAT recommendation was
    the basis for the RFP requirements. It is our contention that this
    work precludes MEDSTAT from bidding on the AHRQ-02-005 Patient
    Safety Database. We would like AHRQ to clarify their intention to
    allow the contractor responsible for the requirements definition to
    be a bidder as a Prime or a Sub.

**Answer: See response to #27 and 28 above.**

30) We understand that MEDSTAT has one or more contracts with IOM,
    perhaps including the Patient Safety Data Guidance effort. Page 12,
    Section 1.3.1 requires the successful Offeror to coordinate with IOM
    (and presumably their contractors). As such, we assume that MEDSTAT
    is ineligible to bid on this contract, either as a prime or
    subcontractor. Is this assumption correct?

**Answer: See response to #27 and 28 above.**

31) Section L.9 Technical Proposal Instructions allows for other-than
    double-spaced text in certain cases. Can single-space text be used
    in tables?

# Answer: Yes

32) Section L.9 Technical Proposal Instructions requires the use of 11
    point pitch. May 10 point pitch be used in tables, and smaller font
    used in graphics, so long as the results are legible?

# Answer: Yes

33) Are there standard COTS products (ETL tools, RDBMS, middleware,
    etc.) that the government would prefer to use for development and
    operation of the system?

**Answer: The database should be either Oracle (version 9i or later), or
MS SQL**

**Server, (version 2000 or later). The Government would prefer a J2EE**

**middleware environment. The Government is equally concerned with any**

**chosen middleware vendor\'s stability and market presence long-term.
There are**

**no standard ETL tools in use at this time.**

34) Since AHRQ is hosting the server for the system, will AHRQ also pay
    under separate arrangement for licensing any COTS software that is
    used in the operation of the system?

**Answer: Yes, within reason. There are many possible development tracks
that**

**could significantly impact the licensing costs. The Government will
need input**

**and discussion with the contractor after award to insure the best
possible choices**

**for all concerned parties.**

35) Are there quality of service (QOS) and availability requirements for
    the delivered system?

Answer: The common web system should be available 24x7, with no more
than 6

hours of scheduled maintenance in any calendar month.

36) The \"NRC\" is listed as an non-DHHS event reporting system in the
    Background under \"Phase III\". What does NRC stand for?

## 

## Answer: Nuclear Regulatory Commission

37) The RFP states that "Offerors should list and summarize any
    contracts (state or federal) or grants (state, federal, or private
    foundation) recently completed (within the last 3 1/2 years - since
    January 1, 1999), or that are currently in process, and describe the
    relevance to the tasks, sub-tasks, and associated activities that
    may be performed under this contract." This appears to create an
    undue burden for submitters, as this would potentially entail
    summarizing thousands of contracts for a contractor team. Should
    this be modified to reflect only relevant contracts completed or in
    process?

**Answer: Since the language above request contracts to describe the
relevance to the tasks, etc., AHRQ is only interested in relevant
contracts completed or in process.**

38) Because of inconsistencies in the MedStat report versus the RFP,
    would you clarify whether VAERS and MedSun are primary integration
    candidates?

## Answer: VAERS and MedSun are primary integration candidates.

39) Who will review members of the ICD-10-CM workgroup? Will workgroup
    members be subject to approval by the PSTF? How many people must
    comprise this group?

> Answer: The PSTF coordinators will review the members to participate
> in the workgroup and the entire PSTF will approve their
> recommendations. The number of people to comprise this workgroup would
> be a minimum of five and a planning maximum of 10 individuals who
> would meet at least twice.

40) The timeline for completion of the IOM\'s Patient Safety Data
    Guidance report is October 2003. Phase I for this project is slated
    to be completed in two years. Would you please clarify how the
    contractor can wait one year into the project to implement the
    standards that will be used in this system?

Answer: Initial development will require the contractor to proceed
without the

benefit of the IOM report. Close liaison will be maintained by AHRQ
between

the two separate projects for maximum integration.

41) You have indicated that CDC's National Healthcare Safety Network
    (NHSN) is a primary integration candidate, yet the system appears to
    be far from operational. Would you please clarify when NHSN will be
    operational and what will be the impact on the overall project
    timeline if NHSN is not complete prior to the commencement of the
    project?

Answer: The final stages of the NHSN are to be included as part of this

contract.

42) It appears that the nation-wide implementation of root cause
    analysis at the hospital level would be extremely time consuming and
    expensive. Would you please clarify the expectations for root cause
    analysis? For example, is it to be on the scale of the MERS-TM
    system?

Answer: The expectation is that the results of root cause analysis would
be

reported and coded in a standard format. It would be up to reporting

institutions on how they would implement their own casual analysis.
MERS-TM

would be a working example of the scale intended.

43) Will there be more than one data entry point for the integration
    candidate databases or will the common web interface become the sole
    point of data entry?

## Answer: The web based entry point is primary.

44) How will the mapping of the data to the extant databases be
    determined? Will the contractor design new business logic or will
    they be expected to use the existing business logic of the primary
    integration candidates? For example, is it expected that information
    sent via paper forms or fax materials will to be accepted in
    parallel, once the Web front end is developed?

## Answer: The contractor should develop the new business plan as part of the plan for Phase I to be approved by the PSTF.

45) Who will be building the electronic interface for the databases?
    Currently electronic interfaces are in different stages of
    completion for the integration candidates, will the contractor be
    expected to design and standardize these interfaces?

## Answer: Yes, the contractor will responsible for building the electronic interfaces for the existing databases of the agencies included in Phase I.

46) Is there a page limitation for the past performance volume (Section
    B)?

Answer: No. Only relevant past performance is requested.

47) In paragraph of task 1."Specific Requirements", there is a direction
    that the contractor shall "... provide linkages to other non-DHHS
    organizations such as other federal reporting systems of the DoD NRC
    and VA, state departments of health and other countries" Please
    clarify if the establishment of these linkages to non-DHHS
    organizations are a required deliverable as part of Phase I
    activities?

> Answer: The linkage to non-DHHS systems is an activity that is
> anticipated to be developed during Phase III of the total project and
> is not within the scope of work for Phase I. However planning for the
> linkage of non-DHHS systems are part of the planning process included
> within the scope of work for this contract.

48) In section 1.1, the statement "...deliver the plan for Phase I to be
    reviewed by the contractor's External; Advisory Panel and the PSTF
    for approval" is included. Please clarify whether this statement
    implies that the EAP will meet to approve the Phase I plan? If yes,
    will the government provide nominations for panel members within 8
    weeks of the Effective Date of Contract?

> Answer: Yes, the EAP is expected to meet and approve the Phase I work
> plan. Yes, the government will provide the recommendations for the EAP
> within the 8 week time requirement.

49) In Program Management sections 2.11, does the requirement to "submit
    implementation plan" occur at 20 months or 18 months? Please clarify

**Answer: 18 months**

50) Does Delivery Schedule 10 also include Phase 3 schedule? Task 1.19
    refers to Plan for Phase 2 and Phase 3.

> Answer: A planning estimate for a schedule is all that is required for
> the plans for Phase II and III.

51) In Section 1.17 please clarify is the "Nation-wide rollout" a Phase
    2 or Phase 3 task? According to the delivery schedule, testing is
    completed at Month 22 of Phase I.

Answer: National roll out is for the front end as part of Phase I.

52) Can years of experience be substituted as equivalence for academic
    requirements? If so, please define what experience may be
    substituted and for what positions.

## Answer: No

53) Section L.9, Item 4 (2) Qualifications of proposed staff: Do you
    mean 1/2 of the staff must have all of these items or across ½ of
    the staff these items must be covered.

Answer: Please refer to the answer provided to question #26.

54) Please define \"outside agencies\". Does this mean outside DHHS?

Answer: Outside agencies refers to agencies outside of DHHS whether they
be

other federal agencies, state, or non-governmental.

55) Page 74, Sec. L.9, Staff qualifications: First item under 2), what
    word is missing between "regularly" and web"?

## 

## Answer: The word should be regulatory not regularly.

56) Pg 82, Evaluation Criteria, Technical Approach: The last bullet
    under Technical Approach ends with the word "and." Is the list
    complete or is there additional information missing?

Answer: This list is complete. Please remove the word "and" at that end
of that

section.

57) Pg 10-11: Is input only coming from hospitals?

## Answer: No. Input can come from any health care facility.

58) The VAERS database system is mentioned in a couple of places in the
    Statement of Work and is included in the data warehouse prototype.
    It is not in the original list of four key databases. Please clarify
    if VAERS is to be integrated into the final product.

## 

## Answer: Yes

59) Are 11x17-inch foldout charts, counting as 2 pages, acceptable in
    the technical proposal?

## 

## Answer: Yes

60) SOW, Task 1.13 User-based software: Will this version of the front
    end include an unpopulated, but fully functional database, or will
    the front-end be applied to the institution's existing database (or
    will it start with the prototype data warehouse)? Is this the system
    that will be tested in 1.15?

Answer: User-based software will include a version of the front end and
an

unpopulated, but fully functional database which local institutions may
use for

their own systems. Any linkage to their existing systems should be up to
the

local institution.

61) Does the web application development need to meet any DISA
    requirements such as network infrastructure?

**Answer: The project must meet HHS and AHRQ security requirements
which**

**will be furnished after contract award.**

62) Are there any standard web development tools and operating systems
    that we need to comply with?

**Answer: J2EE and XML are the middleware development tools of choice.
We**

**will not specify any one vendor, but reserve the right to approve
the**

**development tools chosen.**

63) Is hardware and software cost to be included in our pricing?

## 

## Answer: Yes

64) In the Data Warehousing Prototype section, it states "The five
    snapshots should be moved onto a single analytic platform, combined
    and analyzed". What is meant by "combining" data for data warehouse?
    Isn't this same as integrating all source data to a target standard
    format and consolidated data store using ETL capability, which is
    planned for Phase II? Or does combining mean still have separate
    databases for each source on a central server?

Answer: The pilot data-warehousing prototype is intended to be a test of
the

integration of the five reporting systems in a combined manner. This is
a

prototype to be tested. The results of the prototype testing will
provide valuable

experience and insight for the actual integration that is to occur in
Phase II.

Further, the intent is to have one integrated database, not house
disparate

databases on a single server.

65) Do users need OLAP type capability for the prototype data warehouse?

> **Answer: No, but any chosen technologies should be able to support
> future integration with OLAP tools and OLAP database technology.**

66) Is data cleansing required for the data warehouse (for example,
    patient name and address scrubbing)?

## 

## Answer: Yes

67) Based on the source systems' security, do we need to populate and
    maintain user accounts during Phase I?

## 

## Answer: Yes 

68) Do we need to implement PKI or SSL?

**Answer: SSL - YES. PKI use is to be determined.**

69) Who is going to host the servers (AHRQ)?

## 

## Answer: AHRQ will host the final production server. The contractor is 

## responsible for development and testing servers. 

70) How many users are expected for each phase?

## 

## Answer: These are to be developed by the offeror from estimates of the number 

## of reports estimated in the statement of work in the RFP.

71) What is the interface requirement between the agency web page and
    five source systems (ODBC, Native SQL or XML)?

**Answer: The Government believes XML will be the tool of choice,
however, this**

**will be determined by input from Government technical representatives,
in**

**conjunction with contractor personnel after contract award.**

72) Is there a required or preferred database management system, such as
    \"Oracle\", \"SQL Server\", or \"Teradata,\" with which to implement
    the prototype data warehouse?

**Answer: Oracle or MS SQL Server.**

73) Does AHRQ have any required or preferred query, reporting, and OLAP
    tools?

**Answer: No.**

74) What database management systems are used by the five databases to
    be combined in the prototype data warehouse?

## Answer: See Medstat report vol. 1 and 2., Any further details will be provided after contract award.

75) What technical documentation, such as graphical data models, data
    dictionaries, rules and constraints and triggers and stored
    procedures, is available for the five source databases to be
    combined in the prototype data warehouse?

> **Answer: See Medstat report vol. 1 and 2. Any further details will be
> provided after contract award**.

76) Does AHRQ have a preferred architecture for this project? (.NET/J2EE
    / ...)

**Answer: J2EE**

77) What is the preferred Operation System? (Windows 2000/Linux/Unix)

**Answer: Windows 2000**

78) Does AHRQ have a preferred database? (MS-SQL/Oracle/Sybase...)

**Answer: Oracle or MS SQL Server**

79) What is the present platform/architecture of CDC and FDA event
    reporting systems?

> **Answer: See Medstat report vol. 1 and 2. Any further details will be
> provided after contract award**

80) Ref. Section L.6 of the AHRQ-02-0015 RFP, the following question is
    submitted: Sections C.1.0, C.1.5 , and C.1.12 of the PSC Statement
    of Work (Section C) specify that VAERS is to be included in the
    scope of Phase I, yet the "Background" information of Section C
    omits VAERS from the list of systems in Phase I. Is VAERS in the
    scope of Phase I or not in the scope?

## 

## Answer: Yes. VEARS is in the scope of Phase I.

81) We would like to Partner with a Healthcare consulting company to
    execute this project. In this case, what percent of the work can our
    partner do? Are there any other conditions to be met in this
    partnership?

Answer: See the answer to \# 12 above.

82) Does HHS have any preference/reservation for the technology to be
    used for the Web Interface for example either Java --Based (Java
    Applet, Servlet, JSP, etc.) or Microsoft technology (ASP, COM, MTS,
    .Net, etc.)

> **Answer: Any interface must take into account Section 508
> accessibility rules
> (http://www.access-board.gov/sec508/508standards.htm). As such, Java
> Applets are usually precluded from use since these are not accessible
> through the use of assertive technologies. Other than 508
> considerations, the next primary concern is security, and not one
> specific technology or vendor solution**.

83) Can you let us know the technology used for the following systems
    which are to be integrated:

-   Adverse Events Reporting System (AERS)

-   Vaccine Adverse Event Reporting System (VAERS)

-   Biological Product Deviation (BPD)

-   Manufacturer and User Device Experience (MAUDE)

```{=html}
<!-- -->
```
-   CDC System: National Health Safety Network (NHSN)

> Answer: See Medstat report vol. 1 and 2. Any further details will be
> provided after contract award

84) Page 18; 1.18 Hardware Requirements: Will AHRQ purchase all
    necessary hardware and software packages necessary under this
    program directly? Will the HW/SW be installed by providing vendors?

Answer: AHRQ will provide production hardware. Development and testing

hardware is also required, and is a vendor/contractor responsibility.
The

contractor will purchase and or develop the software.

85) Does AHRQ have standard toolset currently used for other data
    warehousing projects?

**Answer: No**

86) Past Performance Questionnaires: Does AHRQ intend for the responding
    contractor to provide questionnaires to Past Performance references
    or will AHRQ provide questionnaires based on input from the
    proposal?

Answer: The contractor should forward the questionnaires to Past
Performance

references who must complete the forms and return them to the address
stated

in Section L.10 of the RFP.

87) Page 10 - Objectives -- What are the demographics of each of these
    systems? Size, complexity, \# of users, \# of transactions,
    technical environment, etc?

## 

## Answer: See Medstat report vol. 1 and 2. Any further details will be provided 

## after contract award

88) Page 10 -- Specific Requirements -- What documentation will be
    available to the contractors?

Answer: The documentation that is currently used by the various agencies
for

the systems involved will be provided following award of the contract.

89) Page 13 -- Section 1.3.1 -- If the PSDG are not available until
    10/2003; what guidelines will the contractor use in conducting the
    initial tasks under this contract?

**Answer: The guidelines and parameters outlined in the scope of work
contained**

**in the RFP. The Project Office in AHRQ will provide close liaison with
the IOM**

**project and this project and will develop necessary guidelines as
needed and**

**required after award of the contract.**

90) Page 13 -- Section 1.3.2 -- How do we obtain the info/data for
    classifying when an event occurred?

Answer: This information should be provided by the reporting institution
with

field definitions provided in the input screen format.

91) Page 15 -- General -- Did AHRQ release the Sources Sought Notice for
    this solicitation? If so, who responded. How many forms does AHRQ
    expect to exist between all of the systems? For each of the systems?

Answer: AHRQ issued the source synopsis on 5/16/02 in the FedBizzOpps. A

bidders list was not maintained.

A single form is required with subsets for each of the separate systems

92) Page 17 -- General -- What database resources will be available to
    the contractor?

Answer: There is no set of resources which will automatically be made

available. Resource requests will be considered on an as-needed basis
and based

upon their availability.

93) Page 17 -- Section 1.12 -- Explain what is meant by case-based
    reasoning?

Answer: Case-based reasoning is a means of acquiring patterns and
linkages of

events contained within the database that share common or closely
related

features. This form of analysis is looking for common characteristics of
event

reports that share common properties not otherwise identified.

94) Page 18 -- Section 1.13 -- How is this requirement different from
    the common web interface? Who is the user in this case?

Answer: See answer to question 60. The user is the reporting institution
and

hospital or other healthcare institution.

95) Page 20 -- Section 1.20 -- Explain what information and
    documentation will contribute to the evaluation.

Answer: Information and documentation for evaluation would include the

original proposals and any modifications that were made to the project
plan,

monthly reports and results of prototype testing, EAP meeting minutes
and

reports.

96) Page 5 -- Section B.3 -- Does this limitation apply to consultants
    only and not to subcontractors and employees who would be covered by
    the maximum salary limitation addressed elsewhere in the RFP?

Answer: This limitation is for consultants only that are not employees
of the

contractor or a subcontractor.

97) Pages 8-10 -- C-Background, Phase I; Specific Requirements -- This
    section states there are 4 systems we need to develop a common user
    interface for: AERS, BPD, MAUDE & NHSN. Specific requirements,
    states "the contractor shall develop and implement a common
    web-based user interface to integrate the reporting of five current
    reporting systems: NHSN, AERS, BPD, VAERS and MAUDE/MedSun. Are we
    correct in assuming all 5 systems shall be incorporated?

Answer: NHSN, AERS, BPD, VAERS and MAUDE/MedSun are the systems to

be included which make 5 instead of four.

98) Page 12 -- Section C.1.3.1 -- Will AHRQ facilitate access to draft
    copies of the Patient Safety Data Guidance to be published by IOM in
    advance of the anticipated October 2003 publication date?

Answer: AHRQ will facilitate a close liaison with the IOM project, which
may

or may not include actual drafts of the IOM report.

99) Page 17 -- Section C.1.12 -- Will AHRQ facilitate a relationship
    between the winner of this contract and the owners of NHSN, AERS,
    MAUDE/MedSun, VAERS and BPD to resolve data definitions, data
    formats and related issues?

> Answer: Yes. There will also be project coordinators from each of the
> participating agencies who will provide liaison within their
> respective organization.

100) Page 19 -- Section C.1.19 -- This section mandates development of a
     detailed plan for the development of Phase II and Phase III. This
     section goes to state "This plan shall include an update and
     modification of the original plans contained in the contractor's
     initial proposal". Can AHRQ please provide guidance regarding the
     specific plans and the level of detail expected in the contractor's
     proposal for Phase II and III?

Answer: No other details can be provided other than which is provided in
the

RFP. However, the expectation is that specifications for the development
of

Phase II and Phase III will be much clearer and more productive after
Phase I is

designed and has begun to be tested.

101) Page 17 -- Section C.1.13 -- This section states the contractor
     shall develop a user version of the front-end entry system and the
     prototype data warehouse. How will this user-based software be
     different from the common web front-end and the prototype data
     warehouse?

## 

## Answer: See Answer to question 60.

102) Page 17 -- Section C.1.12 -- What is the scope of the prototype
     data warehouse? Should this prototype data warehouse include all
     the data elements of the four operational data stores (i.e. unit
     level data warehouse)?

## 

## Answer: See Answer to question 60.

103) Page 19 -- C.1.19.2 - Please provide examples of the kind of
     reports (Standard or ad hoc) required to query the data warehouse.

**Answer: There are no standard formats or examples available at this
time. It is**

**anticipated that reports would be in a standard set of formats to be
developed by**

**the contractor.**

104) General -- A great deal of the preliminary work for the Patient
     Safety Database (e.g., Attachments 3 and 4 to the RFP) were created
     by MEDSTAT and, in fact, these reports are to serve as a guide for
     the development of a response to the RFP. This long-term
     involvement obviously gives MEDSTAT a tremendous advantage in this
     solicitation since they were aware of details, relationships, and
     nuances not contained in the attachments to this RFP. We
     respectfully request that MEDSTAT be precluded from participating
     in this solicitation to ensure a fair competition.

> **Answer: See response to \# 28.**

105) General -- In order for us to accurately bid the work involved,
     please provide details regarding the hardware, software,
     programming languages, lines of code, database management systems,
     and database size used on each of the NHSN, AERS, BPD, VAERS, and
     MAUDE/MedSun.

Answer: See Medstat report vol. 1 and 2. Any further details will be
provided

after contract award.

106) From reading Section L.9, we understand that there are requirements
     for the following labor categories: One WebDeveloper, Two Analysts,
     One Data Security Personnel and One Business Analyst with Patient
     Safety experience. Please advise on whether there are requirements
     for other labor categories like Database Analyst, Programmer, etc.
     Will the Government confirm that the five personnel will be
     adequate for the performance?

**Answer: The number of personnel needed to adequately perform the scope
of**

**work required is up to the offeror**.

107) We request the Government to relax the specific patient safety
     experience requirement and allow experience gained in any health
     care related area.

## 

## Answer: These requirements are considered essential.

108) The Program Manager is required to have experience in health care
     coding and analysis systems. We request that the government relax
     this specific requirement and allow experience in health care
     related activities, not necessarily "health care coding and
     analysis".

## 

## Answer: See response to \# 26.

109) Reference: Section B.3. Will the Government provide information on
     the travel requirements (number of people, days of stay, etc.) to
     enable the offeror to estimate the travel cost realistically or is
     it assumed that all necessary travel expense during the performance
     period will be allowed?

Answer: Travel estimates are up to the offeror. However, allowance
should be

made for a minimum of two meetings per year for the EAP (15 individuals
with

an overnight stay and the ICD 9/10 working for up to meet at least twice
with

overnight stays). Site visits to the various agencies should be included
as needed

and required by the project plan submitted.

110) Reference: Section M. Will the Government provide the evaluation
     criteria for the Business Proposal? Please let us know how the cost
     proposal will be scored with points.

Answer: AHRQ will perform a basic cost analysis of the proposed cost to
be

submitted in the detail required by Section L.11 of the proposal. The
cost

proposal is not scored with points.

111) Under Section 1 of the Description/Specification Work Statement,
     what exactly does "linkages" mean in the sentence " provide
     linkages to other non-DHHS organizations..."? Is there programming
     involved or is this just a URL linkage?

Answer: Linkage means that the data structure of the DHHS systems should
be

able to link to non-DHHS systems. This linkage is the major scope of
work for

Phase III of the overall effort and is not part of the Phase I which is
the thrust of

this solicitation.

112) Under Section 1.4, what is meant by "computer certificate-based
     system"? Are

we talking about the OS authentication or the Networking authentication?

**Answer: The section is specifying that the common web server must have
a**

**means to pass/accept appropriate authentication credentials to the
backend**

**systems. The Government realizes that this is not a straightforward
task, and**

**will work with the contractor to develop a viable approach.**

113) Under Section 1.7, when should confirmation be sent?

> a\. When Event is saved to Database?

b\. When Event is interfaced over to the appropriate system(s)?

c.  When Event is received by the appropriate system(s)?

**Answer: When received.**

114) Under Section 1.12 (Data Warehouse Prototype)

a\. Are there complete and up to date Data Dictionaries for all 5 of the
following systems: NHSN, AERS, MAUDE/MedSun, VAERS and the BPD?

#### Answer: Yes

b.  Will the prototype be integrated or will it just contain the 5
    databases within one?

##### Answer: It will be integrated as one.

c.  Is the Data Warehouse Prototype to be loaded only 1 time?

##### 

##### Answer: As many times as needed to test the system.

e.  Are you expecting a true dimensional model for the Prototype?

##### 

##### Answer: Yes

f.  Are there to be standard reports or will all reports be ad-hoc?

> **\
> Answer: Standard reports are to be determined by the contractor with
> AHRQ approval.**

115) Under Section 1.13 (User-Based Software)

a\. A data warehouse is typically non-updateable; does this section say
that updates will be done to the data warehouse prototype?

#### Answer: For the user-based software there is to be a program shell.

c.  Why will users want their "own version" of the software. Won't this
    create redundant data?

Answer: No. This will be a way for the local institutions to use the
format and structure of the national system for their own local needs.

116) Under Section 1.16 (Revised the system based on testing) -- Will
     there be a committee for governing which changes are to be done?

## Answer: Yes it will be the EAP and the PSTF.

117) Under Section 1.19 (Plan for Phases II and III) -- What exactly is
     expected in the "detailed plan"? Is this a project plan,
     specification document, etc.?

## 

## Answer: Yes. This is what is expected.

118) The synopsis state that, "The PSTF contracted with the MEDSTAT
     Group, Inc. for an implementation planning study which recommended
     a phased integration process with three distinct phases." AHRQ has
     adopted this acquisition strategy for this procurement. Is MEDSTAT
     Group, Inc. under any Organizational Conflict of Interest (OCI)
     restrictions preventing them from participating as part of a team
     responding to this solicitation?

> **Answer: See response to \# 28.**

119) Section L.9.3 Organizational/Corporate Experience requires 5 years
     experience in certain areas and 3 years experience in other areas.
     Must the Prime contractor have the experience in all of these areas
     or are members of the offeror's team (i.e. subcontractors) allowed
     to fulfill some of these requirements?

Answer: These skills are to be available for the entire project team
either with

the prime or subcontractor.

120) In Phase I, it appears that ICD 9/10 codes may be modified.
     However, this system is designed to work with existing databases.
     Therefore, the front-end may have new codes while the back-end will
     still have the old codes. Will the contractor be required to update
     the codes in the existing databases, or will any modified codes
     only be accommodated when the existing systems are integrated?

**Answer: Modifications to existing systems will take place as part of
Phase II**

**integration and not part of the scope of work for Phase** **I**.

121) In Section F.3, item 6, the user-based software operates off of its
     own database (the prototype data warehouse). However, the prototype
     data warehouse deliverable is not until 6 months later. Is there a
     different back-end required for the user-based software?

## 

## Answer: No. See answer to question 60.

122) Is MEDSTAT Group eligible to bid on this solicitation, either as a
     prime or a subcontractor, or are they precluded from bidding on
     this phase of the project?

Answer: See response to \# 28.

123) Is MEDSTAT allowed to participate as a subcontractor on this
     solicitation?

## 

## Answer: See response to \# 122.

124) Will AHRQ procure the services of an independent verification and
     validation (IV & V) contractor as part of this program? Does AHRQ
     currently have an IV & V contractor under contractor? If so, who?

Answer: No

125) Which AHRQ program office will be the program/project manager for
     the implementation of Phase I, II and III.

## 

## Answer: This information will be provided at the time of award. No contacts to 

## AHRQ program offices should be made until after contract award.

126) Does AHRQ have a government estimate of cost for services for Phase
     I? Phase II? Phase III? If so, can AHRQ disclose this estimate?

Answer: Each contractor should provide their estimate of costs to do the
work

as described in the RFP and attachments.

127) If a contractor does not have a certified cost accounting system or
     indirect rates, would AHRQ be willing to certify the cost
     accounting systems and establish rates for a bidder under this
     solicitation. Also, will AHRQ lower the evaluation marks of a
     bidder if they do not have established rates and a certified cost
     accounting system?

**Answer: The answer to the first part of the question is no. The
awardee should**

**have an acceptable accounting system that conforms with generally
accepted**

**accounting principles \[FAR 31.201-2(a)(3)\]. Under FAR 31.201-1(b),
the**

**accounting system should be able to identify direct and indirect cost
as set forth**

**in FAR 31.2 overall. An award can be made to a contractor who does not
have**

**established rates. However, the contract shall stipulate that within a
period of**

**time the contractor can establish the rates. If this does not happen,
then the**

**contractor is in default and severable due to its stipulations.**

128) Will bidders be penalized or marked down if their primary offices
     are not located in the Washington D.C. metropolitan or geographic
     areas of AHRQ?

## Answer: No

129) Does AHRQ require any particular technical ISO standards (i.e.
     relational database operators) for the proposed solution?

**Answer: No**

130) Can AHRQ provide an estimate of the eventual database size (e.g.,
     number of records, storage size \[GB or TB\] that the winning
     bidder will have to operate and maintain?

**Answer: The estimates of size of reports are available in the RFP in
section 1.1**

**on pages 10 and 11. The offeror should make his/her own estimates of
overall**

**size of the database to accommodate the number of reports that are
possible to**

**be reported.**