Amirfar et al. BMC Public Health 2011, 11:753 http://www.biomedcentral.com/1471-2458/11/753 decision. However, even though vendor-led training was conducted during the implementation phase, it quickly became apparent that adoption even of a seemingly easier technological implementation than a full electro- nic health record can be difficult, particularly when the methodology used is outside of the standard workflow of a physician [6-8]. When the complexity of imple- menting PDA-based CDSS to providers approached the anticipated difficulty of implementing EHR-based CDSS, we decided to concentrate our efforts on a more com- prehensive EHR in order to gain the added benefits over a simple electronic prescribing software. This initial pro- ject provided us the insights to many of the barriers and complexities of technological implementations in clinical settings such as provider comfort with new and existing technology, software and hardware support and mainte- nance required, length and importance of user training, and provider and staff “buy in” in the process. As a result, we sought to identify an EHR vendor that would collaboratively develop our set of public health requirements. In January of 2006, the DOHMH released a Request for Proposals (RFP) to identify a qualified vendor to pro- vide an electronic health record system to our New York City community-based providers as well as to Cor- rectional Health Services, the City entity responsible for providing health care in the New York City’s jail system. After the release of the RFP, six vendors were invited to provide a live demonstration of their product to the DOHMH product review committee, made up of the Assistant Commissioner of PCIP, Chief of Operations of PCIP, Director of Finance and Operations of Health Care Access and Improvement, and Director of Vendors and Technology Partners. The criteria used in the ven- dor selection process included the size of the company, years the company has been in business, stability of the company determined by a stable leadership team over time, the usability of the product as determined through impromptu case presentations to the vendor, ease of use, adaptability, and the interest and resources available to the company to make the necessary changes that we were seeking. The selection of eClinicalWorks as the awardee of the four-year contract was announced in the fall of 2006. eClinicalWorks was chosen because of their commitment to being our partner in building an afford- able public health-oriented EHR. After selection, we soon began development discussions in which we com- municated necessary public health requirements to the eClinicalWorks developers. Once the EHR vendor had been identified, PCIP defined the necessary public health requirements and categorized them into two development cycles. The first development cycle would produce the CDSS alerts as well as the corresponding order sets which would Page 3 of 8 facilitate providers ordering. In addition, during the first development cycle, a more robust registry system that would allow providers to assess their patient population on demand would be developed. The first development cycle involved three phases, which are pre-development discussions, collaborative development, and quality assurance [see Table 2]. The second cycle focused on developing functionality that will allow the DOHMH to distribute public health alerts during important events, such as infectious disease outbreaks. In parallel to working on the Request for Proposals, PCIP started work on translating the ten TCNY priority measures into actionable CDSS alerts at the point of care. These CDSS alerts have been defined as “an auto- mated process for comparing patient-specific character- istics against a computerized knowledge base with resulting recommendations or reminders presented to the provider at the time of clinical decision making” [9] That is, making known to the physician a patient’s qual- ity measure status at the point of care is the decision support. We expanded this standard function of CDSS by making it actionable for our providers so that not only are providers aware of the applicable alert, they can also address the clinical need with one click of a compu- ter mouse. However, the first step in producing these new alerts was to investigate the existing measures at that time and see how they were consistent with the standards produced by other national and state organizations. Our approach to developing electronic CDSS mea- sures involved consulting existing national bodies as well as internal DOHMH subject matter experts in order to translate their retrospective paper measures to point-of-care electronic measures. We examined exist- ing measures produced by national organizations such as the Physician Quality Reporting Initiative and National Committee for Quality Assurance as well as measures produced by independent national and pro- fessional medical associations such as the U.S. Preven- tive Services Task Force, American College of Cardiology, American Diabetes Association, and Amer- ican Lung Association. Table 2 Phases of Development Cycle Step Pre-development Collaborative Development Quality Assurance Goal Involve stakeholders Build consensus Plan frequent meetings Develop clinical to programmatic translation Ensure alpha testing Provide beta and clinical content testing