U.S. ENVIRONMENTAL PROTECTION AGENCY 

OFFICE OF INSPECTOR GENERAL 

Catalyst for Improving the Environment 

Audit Report 

EPA Needs to Improve Management 

Practices to Ensure a Successful 

Customer Technology Solutions Project 

 Report No. 10-P-0194 

August 23, 2010 
Report Contributors:	 Rudolph M. Brevard 

 Cheryl Reid 

 Scott Sammons 

 Kyle Denning 

Joseph Cook 

Abbreviations 

C&A Certification and Accreditation 

CGO Continuing Governance Organization 

CTS Customer Technology Solutions 

EPA U.S. Environmental Protection Agency 

FDCC Federal Desktop Core Configuration 

IV&V Independent Verification and Validation 

MEO Most Efficient Organization 

OEI Office of Environmental Information 

OIG Office of Inspector General 

PWS Performance Work Statement 

QAE Quality Assurance Evaluators 

QASP Quality Assurance Surveillance Plan 

QCP Quality Control Plan 

SOP Standard Operating Procedure 

Cover photo:	 Hardware ready for use during a Customer Technology Solutions deployment. 

(EPA OIG photo) 
 

 

 

 

10-P-0194 U.S. Environmental Protection Agency August 23, 2010 

Office of Inspector General 

At a Glance 

Catalyst for Improving the Environment 

Why We Did This Review 

The Office of Inspector General 

conducted this audit as a result of 

receiving Hotline complaints 

about desktop deployments. We 

sought to determine whether the 

U.S. Environmental Protection 

Agency (EPA): 

•	 Responded to resolve issues 

identified during Customer 

Technology Solutions (CTS) 

deployment. 

•	 Implemented processes to 

eliminate recurring problems 

with deploying CTS. 

•	 Implemented oversight 

practices for the CTS 

contract. 

Background 

CTS service includes acquiring, 

installing, maintaining, and 

supporting computers and 

network printers. The CTS 

deployment schedule projected 

11,744 computers to be replaced 

in 18 locations across the United 

States. CTS is provisioned 

through EPA’s Working Capital 

Fund and provides and 

coordinates all information 

technology end-user support and 

services for Headquarters and 

field offices. 

For further information, contact 

our Office of Congressional, 

Public Affairs and Management 

at (202) 566-2391. 

To view the full report, 

click on the following link: 

www.epa.gov/oig/reports/2010/ 

20100823-10-P-0194.pdf 

EPA Needs to Improve Management Practices to Ensure 

a Successful Customer Technology Solutions Project

 What We Found 

Although EPA indicated it could avoid spending more than $115.4 million 

over 8.5 years by consolidating the desktop computing environment, 

improved management practices are needed to ensure this cost avoidance 

is realized. CTS scheduled replacing 11,744 computers in 12 months, and 

also began providing maintenance and support for those computers and 

attached network printers, without sufficient planning to ensure ongoing 

success. This lack of planning has led to questions about (1) the quality of 

the helpdesk supporting the project; (2) a quality management program 

that is not finalized; (3) key business processes to support ongoing 

operations not being defined; and (4) vacant leadership positions needed to 

facilitate communication and coordination with customers about CTS 

equipment deployments. These conditions resulted in many end users 

voicing dissatisfaction with the CTS deployment and continued 

dissatisfaction with helpdesk support. 

The CTS technical proposal indicates the consolidated desktop services 

solution will result in increased customer satisfaction and provide the 

technical tools and training needed to support EPA’s critical programs. 

However, missteps in project planning led to sustained negative customer 

acceptance of CTS. Without CTS improving its service quality, end users 

may foster a degree of skepticism that will be hard for EPA to overcome. 

What We Recommend 

We made various recommendations to the Director of the Office of 

Technology Operations and Planning, Office of Environmental 

Information, to include: 

•	 Implementing and maintaining a helpdesk that responds to and 

resolves issues in a manner that meets performance metrics; 

•	 Improving the CTS Quality Management Program by developing 

and implementing an Independent Verification and Validation 

process and finalizing the Quality Assurance Surveillance Plan; 

•	 Documenting milestones for the completion of overdue and future 

business process documents; and 

•	 Filling vacant CTS leadership positions. 

In general, the Agency agreed with the findings and recommendations. 
 

 

 

 

 

UNITED STATES ENVIRONMENTAL PROTECTION AGENCY 

WASHINGTON, D.C. 20460 

THE INSPECTOR GENERAL 

August 23, 2010 

MEMORANDUM 

SUBJECT:	 EPA Needs to Improve Management Practices to Ensure a Successful 

 Customer Technology Solutions Project 

 Report No. 10-P-0194 

FROM: Arthur A. Elkins, Jr. 

Inspector General 

TO:	 Vaughn Noga 

Director, Office of Technology Operations and Planning 

Office of Environmental Information 

This is our report on the subject audit conducted by the Office of Inspector General (OIG) of the 

U.S. Environmental Protection Agency (EPA). This report contains findings that describe the 

problems the OIG has identified and corrective actions the OIG recommends. This report 

represents the opinion of the OIG and does not necessarily represent the final EPA position. 

Final determinations on matters in this report will be made by EPA managers in accordance with 

established audit resolution procedures. 

The estimated cost of this report – calculated by multiplying the project’s staff days by the 

applicable daily full cost billing rates in effect at the time – is $177,268. 

Action Required 

We have closed this report in our audit tracking system based on your response to the draft 

report. We believe the proposed actions, when implemented, will adequately address the 

report’s findings and recommendations. Please provide updated information in EPA’s 

Management Audit Tracking System as you complete each planned corrective action or revise 

any corrective actions and/or milestone dates. If you are unable to meet your planned 

milestones, or believe other corrective actions are warranted, please send us a memorandum 

stating why you are revising the milestones or why you are proposing alternative corrective 

actions, as required by EPA Manual 2750. 
We would like to thank your staff for their cooperation. We have no objections to the further 

release of this report to the public. This report will be available at http://www.epa.gov/oig. 

If you or your staff have any questions regarding this report, please contact Rudy Brevard at 

(202) 566-0893 or brevard.rudy@epa.gov. 
 

 

 

 

 

 

 

 

 

 

 

 

 

EPA Needs to Improve Management Practices to Ensure a 10-P-0194 

Successful Customer Technology Solutions Project 

Table of Contents 

Chapters 

1 Introduction ....................................................................................................... 1 

Purpose ...................................................................................................... 1 

Background ................................................................................................ 1 

Noteworthy Achievements .......................................................................... 2 

 Scope and Methodology.............................................................................. 2 

 Prior Reporting ........................................................................................... 3 

2 CTS Helpdesk Lacks Quality............................................................................. 4 

Helpdesk Responsiveness Is Lengthy ....................................................... 4 

Technicians’ Skill Levels Are Inconsistent .................................................. 4 

Key Helpdesk Processes Are Inconsistent and Lacking ............................ 5 

 Recommendations ..................................................................................... 6 

Agency Response and OIG Comments ...................................................... 6 

3 CTS Lacks an Effective Quality Management Program.................................. 7 

CTS Quality Assurance Plan Is Not Finalized ............................................. 7 

Contractor Performance Measurements Are Not Validated by EPA ........... 7 

 Recommendations ...................................................................................... 8 

Agency Response and OIG Comments ...................................................... 8 

4 CTS Business Processes Are Not Defined...................................................... 9 

Key Planning and Operational Documents Are Not Complete.................... 9 

Security Planning Was Not Complete ......................................................... 10 

Recommendations ...................................................................................... 10 

Agency Response and OIG Comments ...................................................... 10 

5 Key CTS Positions Are Not Permanently Staffed ........................................... 12 

Permanent Program Manager Position Vacant During Deployment ........... 12 

Most Efficient Organization Oversight Team Not in Place........................... 13 

 Recommendation ........................................................................................ 13 

Agency Response and OIG Comments ...................................................... 13 

Status of Recommendations and Potential Monetary Benefits.............................. 14 

-continued- 
 

EPA Needs to Improve Management Practices to Ensure a 10-P-0194 

Successful Customer Technology Solutions Project 

Appendices 

A Agency Response to Draft Report.................................................................... 15 

B OIG Hotline Complaint Category Descriptions................................................ 19 

C Distribution ......................................................................................................... 20 
 

 

 

 10-P-0194 

Chapter 1

Introduction 

Purpose 

The Office of Inspector General (OIG) conducted this audit in response to hotline 

complaints about the Customer Technology Solutions (CTS) desktop 

deployments. We evaluated the U.S. Environmental Protection Agency’s (EPA’s) 

implementation of CTS. Specifically, we sought to determine whether EPA had: 

•	 responded to resolve issues identified during CTS deployment, 

•	 put in place processes to eliminate recurring problems with deploying 

CTS, and 

•	 implemented oversight practices for the CTS contract. 

Background 

CTS is an Agency Working Capital Fund service that provides and coordinates all 

information technology end-user support and services for Headquarters and its 

field offices. The CTS service includes acquiring, installing, maintaining, and 

supporting computers and 

network printers. CTS Figure 1-1: Timeframe for Computers Installed 

scheduled 11,744 

computers to be deployed 

in the 12 months from 

60% October 2008 through 

50% October 2008 – September 2009. As January 2009 40% illustrated in Figure 1-1, 

CTS scheduled 30%
 February 2009 

52 percent of these 20% – May 2009 

computers for 10% June 2009 – 

deployment in 4 months, 0% September 

from February 2009 

through May 2009. This 

deployment placed CTS 

equipment in 18 locations Source: OIG compilation and analysis. 

across the United States. 

CTS consolidated the efforts of 22 disparate organizations that provided desktop 

services to EPA into 1 new entity. The Continuing Governance Organization 

(CGO) within EPA’s Office of Environmental Information (OEI) manages the 

new organization. EPA indicated that by implementing CTS it could avoid 

22% 

52% 

26% 

Percentage of Computers
 

Installed In Each 4-Month
 

Period
 

1
 
 

 

 10-P-0194 

spending more than $115.4 million over 8.5 years. CTS is composed of two 

entities working as an integrated Federal/industry team - the EPA Most Efficient 

Organization (MEO) and an industry contractor. The Federal portion of the team 

provides program management, customer relations, quality assurance, and 

contract oversight. The contractor is the operational portion of the team that 

delivers the services. 

Noteworthy Achievements 

The Agency deployed two other simultaneous initiatives: rolling out the Federal 

Desktop Core Configuration (FDCC) standards and upgrading its electronic mail 

software. The FDCC is a new federal policy, which standardizes resources and 

computer settings to increase security and protect government assets. The 

Agency indicated it upgraded its e-mail services to provide enhancements, 

including a more stable platform, faster performance, and the ability to change 

font size. 

Scope and Methodology 

We conducted audit work from April 2009 through May 2010 as a result of 

Hotline complaints of significant problems in deploying the CTS project. We 

performed field work at EPA Headquarters in Washington, DC, and EPA’s 

Research Triangle Park Campus in Durham, North Carolina. We also visited 

Headquarters’ field offices in Las Vegas, Nevada, and the following EPA 

laboratories: 

• National Exposure Research Laboratory in Athens, Georgia. 

• National Air and Radiation Laboratory in Montgomery, Alabama. 

• National Vehicle and Fuel Emissions Laboratory in Ann Arbor, Michigan. 

We performed this audit in accordance with generally accepted government 

auditing standards. These standards require that we plan and perform the audit to 

obtain sufficient and appropriate evidence to provide a reasonable basis for our 

findings and conclusions based on our audit objectives. We believe the evidence 

obtained provides a reasonable basis for our findings and conclusions. 

We reviewed the CTS technical proposal, CTS Performance Work Statement 

(PWS), and quality assurance and quality control documents. We analyzed 

information from 113 OIG Hotline complaints related to the CTS deployment. 

We also analyzed answers from 120 CTS users we interviewed about their 

experiences with CTS. We selected the interviewees using 3 different methods: 

(1) 68 from a CTS user inventory database, (2) 46 according to users' availability 

on the day we visited their location, and (3) 6 from the Hotline complainants. 

Table 1-1 is a breakdown of the number of users that provided feedback by 

source. 

2
 
 10-P-0194 

Table 1-1: Source of CTS Feedback 

Users 120 Interviewed 

Hotline 

Complaints 113 

Source: OIG compilation and analysis. 

Appendix B contains the results of our analysis of the hotline complaints. It 

includes a complaint category table with the number of times each issue occurred 

and category descriptions. Also, we interviewed designated program office points 

of contact; CTS sub-contractors, EPA Zone Representatives, and CTS Service 

Desk personnel. 

Prior Reporting 

In September 2009, we sent a memorandum to the Acting Assistant Administrator 

for Environmental Information. The memorandum conveyed our concerns that 

EPA should deal quickly with our observations from site visits to three EPA 

facilities. We expressed this was necessary to decrease the turmoil evident during 

CTS deployments in an effort to garner users’ acceptance of the new equipment. 

We also issued Quick Reaction Report No. 10-P-0028, Improved Security 

Planning Needed for the Customer Technology Solutions Project, November 16, 

2009. The report conveyed our concerns regarding needing to develop and 

implement a vulnerability testing and remediation process for CTS equipment. 

3
 
 

 

 10-P-0194 

Chapter 2

CTS Helpdesk Lacks Quality 

CTS users voiced dissatisfaction with the quality of CTS helpdesk services. The 

two main helpdesk issues were related to responsiveness and technicians’ skill 

level. Interviews with some CTS helpdesk staff revealed inconsistencies with 

communicating and sharing knowledge, and training. These issues occurred 

because CTS does not have standard helpdesk processes and procedures for use at 

all CTS locations. The CTS Quality Control Plan (QCP) states “To build quality 

into our services, we will establish standardized processes and procedures (SOPs) 

as well as checks and balances (performance monitoring) early in the Phase-in of 

CTS, and train personnel in these measures.” However, CTS did neither of these 

quality-building processes. Unsuccessful helpdesk responses may cause lost 

productivity and down-time for users. Also, it can tarnish a user’s impression of 

an entire organization, such as CTS. 

Helpdesk Responsiveness Is Lengthy 

Monthly CTS performance reports include multiple metrics to measure the 

timeliness of helpdesk response and resolution time. These metrics help to 

encourage quick and efficient operations. However, 57 of 120 users interviewed 

and 51 of 113 Hotline complainants reported a major concern with the 

responsiveness of the helpdesk. Two users reported helpdesk responsiveness 

concerns in both their interview and Hotline complaint. The “responsiveness 

issues” were categorized by users’ complaints of lengthy (1) on-hold wait times 

when calling the helpdesk, (2) response time for a technician to arrive at a user’s 

desk to address an issue, and (3) total resolution time of complaint. One user 

indicated that his email request to the helpdesk did not receive a response for 2 

weeks; another user indicated that the initial helpdesk response was prompt, but it 

took 2 weeks for his problem to be resolved. Helpdesk responsiveness was still a 

concern near the end of the year-long deployment period. Users interviewed 11 

months into the deployment disclosed helpdesk responsiveness remained an issue. 

Technicians’ Skill Levels Are Inconsistent 

The skill level of CTS technicians was a concern, with 23 of 120 users 

interviewed and 16 of 113 Hotline complainants commenting negatively. While 

CTS users overwhelmingly indicated they were happy with their new equipment 

and that the CTS technicians were polite and courteous, many users expressed 

dissatisfaction with the inconsistent results received when technicians responded 

to a trouble ticket. Users’ comments pertained to CTS technicians that were 

either (1) temporary employees tasked with deploying CTS equipment to users, 

(2) temporary employees being considered to fill permanent positions on the 

4
 
 10-P-0194 

helpdesk, or (3) permanent helpdesk technicians. The inconsistency of 

technicians' skill levels in resolving a reported issue also caused delays in the 

users return to work. At one of the CTS sites we visited, a wide disparity of 

experience existed between two temporary technicians being considered for a 

permanent assignment. One technician had more than 10 years of specialized 

information technology experience. The other technician only had one year of 

basic information technology experience. 

Key Helpdesk Processes Are Inconsistent and Lacking 

According to interviews with CTS representatives and technicians, an inconsistent 

approach exists with populating issues and resolutions into a common knowledge 

base. EPA’s April 2009 Service Improvement Plan states that the Agency plans 

to deploy the Knowledge Base component in the helpdesk trouble ticket system. 

This component was supposed to provide a common source for helpdesk 

technicians to retrieve solutions to identified problems. However, implementing 

the component has yet to occur. Some technicians responded they were not aware 

of a common solutions library, while others responded that they used a folder on a 

shared network drive to find resolutions to problems that occurred during 

installments. In addition, a lack of consistency existed in the way technicians at 

different CTS sites shared newly discovered deployment problems with other 

locations. One technician described daily meetings in which issues were shared; 

others said they used emails for sharing, and yet others did not know what the 

proper procedure would be for sharing issues. 

These variations were caused by an inconsistent method for training deployment 

and helpdesk technicians. CTS lacked a uniform training approach across CTS 

deployment sites. For example, at one site deployment staff were given 

classroom training along with on-the-job training by shadowing a more 

experienced technician. At another site, technicians did not receive classroom 

training and were only trained by shadowing more experienced technicians during 

a computer deployment. Furthermore, interviews with helpdesk technicians 

revealed they received limited or no ongoing training. 

In response to the draft report, management indicated they implemented a training 

program followed by all CTS technicians to ensure skills and knowledge remain 

consistent with the needs of the CTS customer base. Also, EPA instituted an 

audit in early 2010 through Office of Administration and Resources Management 

regarding the technical skills and qualifications of the contractor technical staff; 

the audit results defined the skills and qualifications needed for CTS helpdesk staff. 

While management indicated they took steps, more management emphasis is 

needed to ensure that CTS define their business practices and train personnel. 

5
 
 10-P-0194 

Recommendations 

We recommend that the Director, Office of Technology Operations and Planning, 

Office of Environmental Information: 

2-1	 Develop and implement a process to help meet all performance metrics 

related to CTS helpdesk responsiveness. 

2-2	 Document a catalog of all helpdesk processes and train all CTS 

technicians on these respective CTS processes, procedures, and 

technologies. 

2-3	 Implement the use of the Knowledge Base component in the trouble ticket 

system. 

Agency Response and OIG Comments 

In general, the Agency agreed with the findings and recommendations. However, 

the Agency indicated it had an on-board training program for training CTS 

technicians. In the draft report, we recommended that the Agency coordinate with 

the CTS Contracting Officer to better define the qualifications and skill level 

needed for CTS helpdesk staff. In response, CTS management provided us the 

results of an Agency audit of the CTS helpdesk staff's technical skills and 

qualifications that took place during our audit. After reviewing these results, we 

removed our recommendation. Where appropriate, we modified the report. 

6
 
 10-P-0194 

Chapter 3

CTS Lacks an Effective Quality Management Program 

EPA lacks an effective Quality Management Program that monitors all aspects of 

the contractor’s performance. The draft CTS Quality Assurance Surveillance 

Plan (QASP) defines key roles and processes to ensure quality work by the 

contractor. Although this key document was due at the onset of the CTS project, 

EPA has not finalized it, due to delays in establishing agreed-upon details with the 

contractor. As a result, EPA does not have a process to validate performance 

metrics data that are self-reported by the contractor. Without accurate 

performance feedback, EPA management may find it difficult to identify areas 

needing improvement and implement effective solutions. 

CTS Quality Assurance Plan Is Not Finalized 

The CTS QASP dated March 2007 is still in draft as of April 2010. The plan 

describes procedures EPA will use to monitor the Service Providers 

contract/Letter of Obligation performance. The QASP focuses on examining the 

end products and services provided by the contractor. It also describes the 

methodology used to make both quantitative and qualitative evaluations of the 

contractor’s performance. A CTS representative stated that the draft QASP was 

hastily developed to meet Office of Management and Budget Circular A-76 

guidelines for a competitive sourcing proposal. The representative stated that the 

plan still needs to be tailored to fit the CTS model. For example, the CTS 

representative explained that in the CTS model, the roles and responsibilities 

outlined in the plan for Quality Assurance Evaluators (QAEs) are to be filled by 

the Zone Representatives. However, the QASP still lists the positions as QAEs. 

The Agency reported that the QASP will be finalized in early February 2010. 

However, the plan was not complete as of April 2010. 

Contractor Performance Measurements Are Not Validated by EPA 

The validity of performance metric numbers being met each month is uncertain 

because EPA does not have a process in place to validate the measurements that 

are self-reported by the contractor. These measurements include customer 

satisfaction ratings and the timeliness of helpdesk responses and resolutions, 

hardware installs, and incident reporting. Performance measurement reports 

dated December 2008 through September 2009 produced by the contractor show a 

monthly average of 20 out of 44 metrics have not been met. According to a CTS 

representative, some contractor reports were rejected because data appeared 

incorrect. The CTS representative stated the Agency is working with the 

contractor to examine the methodology used for each of the performance 

measures. Also, the Agency plans to develop Independent Verification and 

7
 
 

 

 10-P-0194 

Validation (IV&V) standards and processes for each of the 44 metrics. However, 

as of October 2009, EPA was in preliminary project discussions and lacked a 

process to gather independent performance data or validate the performance data 

being reported by the contractor. Measuring and tracking contractor performance 

is critical to ensuring that the Agency is obtaining contractually required services 

and an indicator that certain types of benefits are, or are not, being achieved (e.g., 

improved user satisfaction). In addition, these data are ultimately used in 

calculating an award fee for the CTS contractor. Without a process in place to 

verify contractor performance, the contractor could receive a monetary award 

without having actually met award criteria. 

Recommendations 

We recommend that the Director, Office of Technology Operations and Planning, 

Office of Environmental Information: 

3-1 Finalize the QASP and verify the plan is tailored to reflect the CTS 

business model. 

3-2 Develop and implement an IV&V process for each of the 44 performance 

metrics. 

Agency Response and OIG Comments 

The Agency agreed with the findings and recommendations. 

8
 
 

 

 10-P-0194 

Chapter 4

CTS Business Processes Are Not Defined 

The contractor has not completed key planning and operational documents that 

outline critical business processes. The CTS PWS cites that the CTS Service 

Provider is responsible for documenting all efforts associated with the PWS, 

including supporting documents for installing, testing, and operating the systems. 

A CTS representative stated insufficient time was allocated to project planning, 

which led to incomplete and untimely deliverables from the contractor. This 

delay occurred because the same contractor staff integral to the CTS deployment 

were also responsible for developing these key documents. Success in 

implementing desktop services management rests in large part on thorough 

preparation and planning before deployment. Without this, significant problems 

could occur during deployment. Further, without complete documentation of key 

business processes, the Agency has no standard in which to measure the 

completeness, quality, and accuracy of the contractor’s products and performance. 

Key Planning and Operational Documents Are Not Complete 

As of January 2010, the contractor has yet to complete key planning and 

operational documents, due in June-July 2008, to EPA standards. The documents 

are: 

• Configuration Management Plan, 

• Phase-In Plan, 

• Program Management Plan, 

• Quality Control Plan, and 

• Standard Operating Procedures. 

While the contractor provided these deliverables, EPA rejected the documents 

because required standards had not been met. A contractor performance 

evaluation cited 

...the content of the documentation was not fit for its intended 

 purpose, which was to describe the processes and procedures 

that would accomplish the overall requirements of the PWS. The 

documents lacked clarity and appropriate details, did not reference 

industry standards cited in the contract, and did not include the 

required schedule for completion of specific tasks in the plan. 

9
 
 10-P-0194 

Also, the contractor has not provided all required recurring reports such as status 

reports, issue tracking reports, and inventory reports. Even though the contractor 

had not completed these documents to the Agency’s satisfaction, the EPA 

contracting office made payment to the contractor for these deliverables. Without 

these key planning and operational documents, management can not verify that 

PWS requirements are being carried out as planned. 

In response to the draft report, management indicated they are using the award fee 

process as a way of imposing consequences on the contractor for nonperformance. 

Although, the purpose of the award fee should to motivate the contractor to meet 

performance metrics it does not include contractor’s performance pertaining to 

developing key planning and operational documents. As such, it is critical that 

management establish milestones for delivering these key documents and internal 

control processes to ensure deliverables are approved before payment. 

Security Planning Was Not Complete 

One of the more critical documents that the contractor did not complete was the 

CTS Security Plan. As with the other overdue CTS documents from the 

contractor, the security plan submitted did not meet EPA requirements and was 

rejected. To move forward with the certification and accreditation (C&A) of 

CTS, EPA had to rewrite the Security Plan (a required C&A document). This 

critical document was not approved until November 2009. Further, the 

contingency planning portion of required CTS security documents was also not 

completed until January 2010. Without security planning documents in place, 

management could not be reasonably assured of the security and availability of 

the system and that Agency operations would continue in the event of a disaster. 

Recommendations 

We recommend that the Director, Office of Technology Operations and Planning, 

Office of Environmental Information: 

4-1 Document milestones for completing overdue and future contractor 

deliverables. 

4-2 Document the process used to verify deliverables are complete and 

accepted prior to paying contractor. This should include withholding 

future payments to contractor until required deliverables are delivered and 

approved. 

Agency Response and OIG Comments 

In general, the Agency agreed with the findings. Management indicated 

milestones have been developed and are in place for the required contractual 

deliverables, and management will document these in the project plan by 

10
 
 10-P-0194 

September 15, 2010. The Agency also indicated it has a process in place to 

ensure deliverables are completed and accepted prior to payment. However, 

management indicated they have not documented the process. Management 

indicated they have an award fee process in place, but an award fee has not been 

paid to the contractor up to this point. Management needs to document the 

processes used for evaluating performance. Where appropriate, we modified the 

report. 

11
 
 10-P-0194 

Chapter 5

Key CTS Positions Are Not Permanently Staffed 

EPA did not have a permanent, full-time CTS Program Manager during the 

12-month deployment. The CTS Continuing Governance Organization (CGO) 

positions remain vacant as of November 2009. The CGO provides management 

oversight of the CTS deployment and continued operations for over 11,700 end 

users. The technical proposal states that the above positions were to be in place 

by October 1, 2007. Problems encountered during the Program Manager hiring 

process resulted in a vacancy of the CGO positions. As a result, deploying over 

11,700 computers was burdened with problems and created a frustrating transition 

for end users. This may result in a sustained negative impression of CTS that the 

Agency will need to overcome for users to accept and support continuing the CTS 

environment. 

Permanent Program Manager Position Vacant During Deployment 

The CTS deployment period lacked a permanent full-time Program Manager, a 

key management position for this project. This MEO team position is responsible 

for activities such as developing policies, performing long- and short-term 

planning, managing customer focused activities, and meeting service performance 

measures. The CTS technical proposal required this position to be in place before 

starting deployment. However, the Program Manager position was filled in an 

acting capacity on a part-time basis until hiring someone full-time at the end of 

September 2009, the last month of the deployment period. The staff member 

filling the role on a part-time basis was also responsible for heading an OEI 

branch during this time. This management official stated that performing both 

roles left an insufficient amount of time available to perform the CTS Program 

Manager duties. 

Two key areas of CTS affected by this lack of time were communications and 

developing processes and procedures for deployment--both key planning areas 

that, if neglected, can negatively impact a project. Agency managers and end 

users stated that better communication was needed and that the CTS deployment 

lacked key procedures and processes. Staff members responsible for coordinating 

with CTS for the computer installations at their location stated that the 

communication plan was not sufficient to make them feel prepared for the 

deployment. They also stated that if they had not taken the lead on preparing their 

office for the deployment, the situation would have been more problematic. Staff 

members also stated that without their proactive and follow-up actions before and 

during the deployment, it would have been more problematic. 

12
 
 

 

 10-P-0194 

Most Efficient Organization Oversight Team Not in Place 

The CGO is a leadership team intended to govern activities of the CTS team. The 

CGO positions were originally planned to be in place by October 1, 2007, before 

the deployment. However, as of November 2009, the four CGO positions were 

still vacant. These positions are Program Manager, Project Officer, Technical 

Architect, and Service Manager. A CTS representative stated that these vacancies 

hindered the deployment. An example given by the CTS representative was 

problems encountered by eBusiness (an Agency system used by CTS for ordering 

computers and related hardware and the billing of this equipment). The 

representative stated that the CGO Service Manager position could have used the 

assistance of eBusiness to avoid many of the problems encountered during the 

deployment. When questioned about the delay in hiring the CGO positions, a 

representative stated that the MEO positions took priority over these positions. 

The lack of guidance that this oversight board could have provided may have 

caused many of the issues and delays in CTS deployment activities. Hiring these 

positions as soon as possible will provide the management oversight and guidance 

needed to assist in reducing and remediating problems during the operations and 

maintenance phase of the CTS project. 

Recommendation 

We recommend that the Director, Office of Technology Operations and Planning, 

Office of Environmental Information: 

5-1	 Give priority to filling all four CGO vacancies, and establish a milestone 

date. 

Agency Response and OIG Comments 

The Agency agreed with the findings and recommendation. 

13
 
 

 

 

 

 

 

 

 

 

 

 

 

 10-P-0194 

Status of Recommendations and 

Potential Monetary Benefits 

POTENTIAL MONETARY 

RECOMMENDATIONS BENEFITS (in $000s) 

Rec. 

No. 

Page 

No. Subject Status1 Action Official 

Planned 

Completion 

Date 

Claimed 

Amount 

Agreed To 

Amount 

2-1 6 Develop and implement a process to help meet all 

performance metrics related to CTS helpdesk 

responsiveness. 

O Director, 

Office of Technology 

Operations and Planning, 

Office of Environmental 

Information 

01/15/11 

2-2 6 Document a catalog of all helpdesk processes and 

train all CTS technicians on these respective CTS 

processes, procedures, and technologies. 

O Director, 

Office of Technology 

Operations and Planning, 

Office of Environmental 

Information 

12/15/10 

2-3 6 Implement the use of the Knowledge Base 

component in the trouble ticket system. 

O Director, 

Office of Technology 

Operations and Planning, 

Office of Environmental 

Information 

01/30/11 

3-1 8 Finalize the QASP and verify the plan is tailored to 

reflect the CTS business model. 

O Director, 

Office of Technology 

Operations and Planning, 

Office of Environmental 

Information 

04/01/11 

3-2 8 Develop and implement an IV&V process for each 

of the 44 performance metrics. 

O Director, 

Office of Technology 

Operations and Planning, 

Office of Environmental 

Information 

02/15/11 

4-1 10 Document milestones for completing overdue and 

future contractor deliverables. 

O Director, 

Office of Technology 

Operations and Planning, 

Office of Environmental 

Information 

09/15/10 

4-2 

5-1 

10 

13 

Document the process used to verify deliverables 

are complete and accepted prior to paying 

contractor. This should include withholding future 

payments to contractor until required deliverables 

are delivered and approved. 

Give priority to filling all four CGO vacancies, and 

establish a milestone date. 

O 

O 

Director, 

Office of Technology 

Operations and Planning, 

Office of Environmental 

Information 

Director, 

Office of Technology 

Operations and Planning, 

Office of Environmental 

Information 

09/15/10 

09/15/10 

O = recommendation is open with agreed-to corrective actions pending 

C = recommendation is closed with all agreed-to actions completed 

U = recommendation is undecided with resolution efforts in progress 

14
 

1 
 

 

 

 

 

 

 

 

 10-P-0194 

Appendix A 

Agency Response to Draft Report 

July 12, 2010 

MEMORANDUM 

SUBJECT: OEI Comments on the Draft Report: EPA Needs to Improve Management 

Practices to Ensure a Successful Customer Technology Solutions Project, Audit 

No: OMS-FY09-0004 

FROM: Vaughn Noga, Director 

Office of Technology Operations and Planning 

And Chief Technology Officer 

TO: Rudy Brevard, Director 

 Information Resources Management Assessments 

Office of Inspector General 

We appreciate the opportunity to review the draft Report, “EPA Needs to Improve 

Management Practices to Ensure a Successful Customer Technology Solutions Project.” 

In general, we found the report was an accurate reflection of the Customer Technology 

Solutions (CTS) service at the time of the audit and activities that will improve the service. We 

concur with some of the findings and recommendations; however, given the duration of time 

between the audit, conducted in April 2009, and the final report, provided in June 2010, many of 

the findings and subsequent recommendations have been accomplished. Please refer to the 

associated Corrective Action Plan for further details. 

If you have any questions regarding this report, please contact me at 202-564-6665, or 

your staff may contact Michael Fays at 202-566-0438. 

cc: Johnny Davis 

 Liza Hearns

 Michael Fays 

15
Office of Environmental Information 

Corrective Action Plan 

Auditing Group: OIG Audit Title: EPA Needs to Improve Management Practices to 

Audit No.: OMS-FY09-0004 Customer Technology Solutions Project (draft) Report Date: June 10, 2010 OEI Lead and Phone: Michael Fays 202-566-0438 

OEI Lead Office: OTOP/EDSD 

Recommendation Corrective Action Planned 

Completion Date 

Status / 

2-1 Develop and implement a 2-1-1 Develop a process to help meet 2010/11/15 2-1-1 PM attainme

process to help meet all performance metrics (PM) related to CTS helpdesk 

PM on responsiveness 

2-1-2 Implement process 2011/01/15 

positive manner as -- April 2009: 47% -- April 2010: 73% 

responsiveness. This has been accom

specific metrics and analysis of metrics cEfforts in this respeformal process will 

2-2 Document a catalog of all 2-2-1 Develop a helpdesk process 2010/12/15 2-2-1 Knowledge ahelpdesk processes. Develop and implement a 

catalog developed based on 

the appropriate doccompiled into a pro

process to train all CTS date. technicians on the respective CTS processes, procedures, and 2-2-2 Develop process to train CTS 2010/10/01 2-2-2/2-2-3 Non-ctechnology. technicians training program is 

CTS technicians to 

2-2-3 Implement process to train technicians 

2010/11/15 remain consistent w

customer base. 

2-3 Coordinate with the CTS Contracting Officer to better define the qualifications and skill level needed for CTS 

Describe work CO and audit has done to define qualifications and skill level of CTS staff. 

2010/10/15 Non-concur. EPA 

2010 through OARskills and qualificatitechnical staff; the rpending. 

16
helpdesk staff. 2-4 Implement the use of the Knowledge Base component in the trouble ticket system. 

Implement the use of Knowledge Base 

component in the Remedy system. 

2011/01/30 

3-1 Finalize the QASP and verify the plan is tailored to reflect the CTS business model. 

3-1-1 Finalize the QASP 3-1-2 Verify the QASP is tailored to 

reflect the CTS business model 

2011/04/01 2010/12/15 

The team plans to frelevant to the CTS work to finalize by 

3-2 Develop and implement an 3-2-1 Develop IV&V process for the 2011/01/15 IV&V process for each of the PMs 44 performance metrics. 

3-2-2 Implement the IV&V process 2011/02/15 

4-1 Establish a milestone for completing overdue CSC deliverables and verify deliverables meet EPA standards. 

Establish milestones in project plan for overdue deliverables. Per schedule in project plan, verify the deliverables meet EPA standards 

2010/09/15 Non-concur. Miles

and are in place for deliverables, and wiindicated date. 

4-2 Determine whether 4-2-1 Determine compensating 2010/08/15 compensating measures are measures needed for key needed until key documents are documents delivered, and if so, establish those measures. 4-2-2 Establish project plan for 

establishing those measures. 

2010/09/15 

4-3 Verify all deliverables are Verify process to ensure deliverables are 2010/09/15 Non-concur. CTS P

complete and accepted prior to paying contractor. 

complete and accepted prior to payment. Team and the Cont

regarding the critericomplete deliverablprocess governing twill be documented 

4-4 Coordinate with the CTS Coordinate with CTS CO to impose 2010/09/15 Non-concur. This Contracting Officer to impose consequences when contractor fails to deliver products and 

consequences for failed delivery. Award Fee process 

utilized and, up to tnot received an AwEPA team continue

services according to contracted and reject contracto

levels on metrics fo

17
timelines. This should include withholding future payments to CSC until all required deliverables are delivered. 

consensus regarding calculation. The acdocumented by the 

5-1 Give priority to filling out all four CGO vacancies, and establish a milestone date. 

Outline priorities for filling CGO vacancies and establish milestone dates for filling vacancies 

2010/09/15 CTS Program Mana

September 2010. Twas filled March 20remaining positions documented, along indicated date. 

18
 10-P-0194 

Appendix B 

OIG Hotline Complaint Category Descriptions 

The table below shows the 113 OIG hotline complaints by category and the number of times that 

each of the six issues was cited. (Note: Some of the 113 complaints had more than 1 issue.) 

Table A-1: OIG Hotline Complaint Categories with Number of Occurrences 

Technician 

Skill Level 

Helpdesk 

Responsiveness Printers “Deployment Day” Issues Hardware Non-CTS 

16 51 21 10 37 42 

 Source: OIG compilation and analysis. 

•	 The “Technician Skill Level” category incorporates users’ comments about CTS 

technicians not possessing the appropriate skill level to complete an assigned task or 

duty. 

•	 The “Helpdesk Responsiveness” category captures complaints related to long on-hold 

times when calling the helpdesk, technicians taking a long time to respond to a help 

request, or a lengthy issue resolution time. 

•	 The “Printers” category incorporates complaints related to new CTS printers (mostly 

issues related to duplex printing) and connectivity issues with existing non-CTS printers. 

•	 The “Deployment Day Issues” category captures complaints of problems only 

experienced on the day of deployment. 

•	 The “Hardware” category captures complaints that the new CTS hardware is “inferior” to 

what existed and slow start-up times on the new equipment. In most cases, slow start-up 

times of new hardware were due to heavy network traffic during deployment and 

improved after the specific site deployment was complete. 

•	 The “Non-CTS” category incorporates problems related to FDCC standards, email 

system upgrade, encryption, and general setting problems. 

19
 
 10-P-0194 

Appendix C 

Distribution 

Office of the Administrator 

Assistant Administrator for Environmental Information and Chief Information Officer 

Assistant Administrator for Administration and Resources Management 

Agency Follow-up Official (the CFO) 

Agency Follow-up Coordinator 

Director, Office of Technology Operations and Planning, Office of Environmental 

 Information 

Director, Enterprise Desktop Solutions Division, Office of Environmental Information 

Director, Office of Acquisition Management, Office of Administration and 

Resources Management 

General Counsel 

Associate Administrator for Congressional and Intergovernmental Affairs 

Associate Administrator for External Affairs and Environmental Education 

Audit Follow-up Coordinator, Office of Environmental Information 

Inspector General 

20