Information Technology: SSA Needs to Address Limitations in Management Controls and Human Capital Planning to Support Modernization Efforts
Published: 2014-05-08T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: SSA relies on IT for delivering Social Security services to virtually every American. The agency reportedly spent about $1.5 billion for IT in fiscal year 2013, and it plans to continue modernizing its aging systems. Management controls and human capital are critical in helping ensure effective and efficient IT project implementation.
GAO was asked to examine SSA's IT modernization efforts. The study (1) assessed selected IT investments to determine the extent to which they adhere to SSA's investment management controls and are improving services and (2) determined how SSA's IT human capital program, including the identification and implementation of critical skills and competencies, is supporting its current and future modernization efforts. To do so, GAO reviewed key management controls for one project from each of five SSA-defined project types, including one project with the highest resources for its type and four randomly selected projects; compared human capital planning documents with relevant guidance; and interviewed relevant SSA officials.
section_title: What GAO Found
paragraphs: The Social Security Administration’s (SSA) selected information technology (IT) projects did not fully adhere to management controls called for by its IT project management guidance, which are essential to effectively oversee and monitor IT investments. Such controls include, among others, a cost-benefit analysis, risk mitigation plan, and project schedule. For the five projects selected, SSA developed the majority of the documents required to demonstrate adherence to management controls; however, most had limitations. One project that was required to complete 11 control documents had developed 5 without limitations, but the remaining 6 had limitations. For example, while certain risks to the project were identified, the documentation did not include risk mitigation plans, which are essential for avoiding, reducing, and controlling the probability of the occurrence of identified risks. Across the five projects, the most common limitations included a lack of traceability (which is needed to track project history and demonstrate that requirements are met) and inaccurate or incomplete information, such as project schedules that had inaccuracies in key milestone dates. The limitations could be attributed to, among other things, IT oversight systems that did not include all needed data or fully support traceability, and a quality assurance process that was not effectively implemented. The agency recently took steps that should help improve its quality assurance process.
Further, while SSA stated that its projects have resulted in improved services, it was not able to demonstrate this. In particular, while three of the five projects identified performance measures, these measures generally were not specific enough to determine projects’ contributions to improved services, and baselines against which to measure improvement were not established. Ensuring that management controls are consistently and effectively implemented would help ensure the efficient use of agency resources.
SSA’s IT human capital program has identified skills and competencies to support certain workforce needs, but lacks adequate planning for the future. The agency has developed IT human capital planning documents, such as its recent Information Resources Management plan and skills inventory gap reports, which identified near-term needs, such as skill sets for the following 2 years. Nevertheless, SSA has not adequately planned for longer-term needs because its human capital planning and analysis are not aligned with long-term goals and objectives and the agency does not have a current succession plan for its IT efforts. The agency has recognized challenges with regard to employee retirements and a recent hiring freeze, which have put constraints on resources for certain investments. While SSA officials stated that an updated human capital operating plan will be completed in June 2014, they could not specify how it would address future IT human capital needs. Until these needs are identified, SSA may lack critical plans for addressing IT resources and skills to support agency-wide IT investment goals.
section_title: What GAO Recommends
paragraphs: GAO is recommending that SSA (1) perform effective oversight to ensure control documents are developed, complete, and accurate and that oversight systems include needed data and support traceability; (2) ensure project control documents identify specific performance measures and baselines; and (3) identify long-term IT needs in its updated human capital operating plan. SSA agreed with GAO's recommendations.

section_title: Letter
paragraphs: The Social Security Administration (SSA) is responsible for delivering  Social Security services that touch the lives of virtually every American.  To provide these services, the agency relies on a variety of information  technology (IT) systems, ranging from those that support the processing  and payment of Disability Insurance and Supplemental Security Income  benefits to those that facilitate the calculation and withholding of Medicare  premiums. For fiscal year 2013, the agency reported that it spent  approximately $1.5 billion for IT.
Many of SSA’s systems have aged and present various challenges to the  efficiency of the agency’s existing IT environment. Consequently, the  agency has committed to, and has initiated numerous efforts toward,  investing in modern technologies needed to update its aging IT  infrastructure and improve services. Its efforts in this regard are to be  guided by management processes and controls that, according to the  agency, exist to help ensure that its investments meet the strategic and  business objectives of the agency.
Given the importance of technology in supporting SSA’s ability to meet its  mission, you requested that we examine the agency’s modernization  efforts. Specifically, our objectives were to (1) assess selected IT  investments to determine the extent to which they adhere to SSA’s  investment management controls and are improving services; and (2)  determine how the agency’s IT human capital program, including the  identification and implementation of critical skills and competencies, is  supporting its current and future modernization efforts.
To address the first objective, we selected a non-generalizable sample of  IT projects, using data files provided by SSA that included investment  data from fiscal years 2010 to 2012. Our criteria considered each of the  project categories that SSA has defined and the different sizes of projects  based on actual work years of effort. Specifically, we selected one project  because it had the most resources (i.e., work years) dedicated to it for  that project category; we then selected four others randomly from the  remaining project categories.
In addition, we analyzed industry best practices and guidance used to  effectively manage IT projects. We also analyzed policies and guidance  reflected in the agency’s IT project management directive and quality  assurance processes to identify the types of control documents, such as  project proposals, cost-benefit analyses, and project schedules, that SSA  relies on for managing its projects. Among those identified, we  judgmentally chose 11 types of management control documents for our  evaluation. These 11 controls were called for in the agency’s IT project  management directive and quality assurance processes, as well as other  external best practices; covered key management processes—select,  control, and evaluate; and spanned each phase of the projects’ life cycle.  In addition, they were represented in each randomly selected project.
Subsequently, for each of the five selected projects, we evaluated the  effectiveness of the controls implemented. To do so, we obtained the  applicable documentation from SSA, where available, and assessed it  against the agency’s guidance and other best practices. Specifically, we  compared the documents to the agency’s IT project management  guidance, as well as best practices outlined in the Software Engineering  Institute’s (SEI) Capability Maturity Model for Integration (CMMI) for  Development (version 1.3), our IT investment management framework,  and our standards for internal control. In assessing the documents, we  identified any limitations in the controls, such as data inaccuracies or  inconsistencies. Further, to determine whether the agency’s IT  investments had improved services, we analyzed information that was  identified in project documentation for measuring the performance of each  project. We also interviewed project managers, project team members,  and other agency officials, including the Deputy Commissioner for  Systems/Chief Information Officer (CIO), to understand the agency’s  investment management processes and its IT project management  guidance and controls.
For the second objective, we obtained and analyzed available human  capital plans and data, including the agency’s Information Resources  Management (IRM) Strategic Plan for fiscal years 2014-2017, 2-year IT  skills inventory gap reports, IT staffing data for fiscal years 2008-2014,  agency-wide succession plan, and agency-wide human capital plan. We  also noted any human capital issues that were identified during our  interviews of agency officials or reviews of documentation supporting the  five selected projects.
We assessed the reliability of the data that we used to support the  findings in this report by reviewing relevant IT investment files and  program documentation to substantiate evidence obtained from agency  databases. We analyzed agency database instructions and replicated the  system processes with SSA data files and compared the results with  agency-provided data. We also corroborated documentation on program  processes and projects obtained from SSA through interviews with SSA  officials. We determined that the data used in this report are sufficiently  reliable.
We conducted this performance audit from November 2012 to May 2014,  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives. Additional details on our  objectives, scope, and methodology can be found in appendix I.
subsections: section_title: Background
paragraphs: SSA’s mission is to deliver Social Security services that meet the  changing needs of the public. The Social Security Act and amendments  established the programs that SSA administers. Among these, the Old  Age, Survivors, and Disability Insurance program—commonly referred to  as Social Security—is one of the nation’s largest federal retirement and  disability programs. Financed by two trust funds, the program provides  monthly benefits to retired and disabled workers, their spouses, children,  and the survivors of insured workers. In addition, Supplemental Security  Income is a needs-based program that is financed from general tax  revenues. It is designed to provide benefits to aged adults and to blind or  disabled adults and children who have limited income and resources.
According to SSA, in fiscal year 2013, about 63 million people received  Social Security or Supplemental Security Income benefits for a total of  about $855 billion in benefits paid. Collectively, about 165 million people  work and pay Social Security taxes, and 88 percent of individuals age 65  and over receive Social Security benefits.
Organizationally, SSA is very large. It is headed by the Commissioner,  who is assisted by the Deputy Commissioner and various other executive  officials, including the Chief and Deputy Chief of Staff, Executive  Secretary, and eight deputy commissioners who are responsible for the  agency’s various business components.
The Commissioner is supported by about 75,000 federal and state  employees who are located at SSA’s headquarters in Baltimore,  Maryland, and throughout a decentralized network of about 1,500 offices  that includes 10 regional offices, 6 processing centers, and approximately  1,260 field offices, as well as teleservice centers, hearing offices, and 54  state and territorial Disability Determination Services offices. In support of  their operations, the offices perform an assortment of interrelated and  interdependent business functions, including operational, policy, financial  management, and legislative relations.
subsections: section_title: SSA’s IT Environment
paragraphs: SSA relies extensively on computer hardware and software to carry out  its core mission functions. In fiscal year 2013, the agency reported  expenses totaling $11.6 billion to support its mission and programs— approximately $1.5 billion of which was spent for IT. Specifically,  computer systems are used to administer programs and support related  administrative needs that include, among others, handling millions of transactions via SSA’s toll-free telephone number; maintaining records for the millions of beneficiaries and other  recipients of the agency’s programs, including Supplemental Security  Income and Old Age, Survivors, and Disability Insurance; evaluating evidence and making determinations of eligibility for processing and posting employer-reported earnings to workers’  benefits on new claims;  issuing new and replacement Social Security cards; processing continuing disability reviews; processing non-disability Supplemental Security Income  redeterminations.
Many of SSA’s existing computer systems  were developed in the 1960s  and 1970s, and while the agency has performed technical and functional  upgrades throughout the years to accommodate legislative and policy  changes, these legacy systems have aged. Accordingly, as they have  aged, the systems have presented various challenges to the efficiency of  SSA’s existing IT environment.
SSA conducts periodic reviews to determine if beneficiaries are still eligible to receive  Supplemental Security Income payments based on the beneficiary’s income, living  arrangement, and other non-medical factors—these reviews are referred to as  Supplemental Security Income redeterminations.
According to the Deputy Commissioner of the Office of Systems/CIO, these systems  largely represent software applications that have been developed; however, the agency  has continued to make technical and business enhancements to these applications.
Many of its programs are written in Common Business Oriented  Language (COBOL), which is one of the oldest computer  programming languages. The agency reported that it runs hundreds  of COBOL applications and that this language has served the agency  for over 40 years. It currently has roughly 60 million lines of COBOL in  production that support the agency’s high-transaction volume and  enable the agency to meet its regulatory, benefit, and reporting  requirements. While SSA officials referenced a study indicating that  the benefits of a wholesale replacement of COBOL would not  outweigh the risks because of the large size and cost of rewriting their  COBOL applications, the Assistant Deputy Commissioner for  Systems/Deputy CIO said the agency has a plan to migrate away  from this language and that new applications are being developed in  more modern programming languages, such as Java.
The National Computer Center is SSA’s facility that currently houses  its nationwide computer operations. Mainframe systems within this  data center maintain critical demographic, wage, and benefit  information essential for providing service to millions of individuals, as  well as other federal, state, and local agencies, daily. However, the  center is over 30 years old and many of its infrastructure systems are  well past their designed life cycle. SSA has expressed concerns that  the center could deteriorate to the point that a major failure to its  systems could jeopardize its ability to handle increasing workloads  without interruption. The agency is in the process of developing a new  data center to replace its existing center in Baltimore, Maryland.
While SSA has efforts under way to convert to a more efficient  database system for its critical mission support files and uses more  modern databases for its new applications, the agency continues to  use its Master Data Access Method database system, which does  not support industry standards for automatic data access. Although it  has converted several files, such as the Master Earnings Record and  Supplemental Security Record, to a more modern database, the  agency has not yet completed the conversion for one of its largest  files—the Master Beneficiary Record file.
SSA stores, processes, and shares increasing amounts of data with  public- and private-sector partners, and the agency faces an  increasing need to transition to web-based online access for its data  and services.
section_title: SSA’s Organization Responsible for IT
paragraphs: The Office of Systems, headed by the Deputy Commissioner for Systems  (who also serves as the agency’s CIO), is SSA’s primary organization  responsible for overseeing the design, development, and maintenance of  the technology resources that support the agency. The Deputy  Commissioner for Systems/CIO reports to the Commissioner of Social  Security. According to the agency, 3,403 IT positions are assigned to the  Office of Systems. In addition, the office relies on contractors to fill  certain skill gaps, where necessary. The office reported that in fiscal year  2013 approximately 1,402 contractor work years were used to support IT  projects.
Staff in the Office of Systems are assigned to eight component offices:
Office of Applications and Supplemental Security Income Systems— responsible for most phases in the systems development life cycle for  the Supplemental Security Income, Quality Assurance, Customer  Help Information, and Representative Payee programs.
Office of Disability Systems—develops, implements, and maintains  electronic systems to support disability programs.
Office of Earnings, Enumeration, and Administrative Systems— designs, develops, and maintains SSA’s earnings, enumeration, and  administrative systems.
Office of Enterprise Support, Architecture, and Engineering—identifies  the strategic IT resources needed to support SSA business processes  and operations.
Office of Information Security—manages and directs the agency’s  overall information systems security program.
Office of Retirement and Survivors Insurance Systems—responsible  for programmatic and management information systems supporting  these programs, as well as for the post-entitlement activities  associated with the Disability Insurance program.
Office of Systems Electronic Services—directs the development of  software that supports electronic service-delivery initiatives.
Office of Telecommunications and System Operations—responsible  for supporting the computer systems and networks infrastructure.
In addition, the Office of Systems is responsible for the IT portion of the  human capital planning that supports agency-wide human capital goals.  For example, the Office of Systems submits an annual Human Capital  Management Report to SSA’s Office of Human Resources.
The report  provides an assessment, analysis, and the results of human capital  strategies, activities, and operations related to IT.
Figure 1 provides a simplified organizational chart depicting the agency’s  IT organization and component offices.
section_title: SSA’s Management of IT Investments
paragraphs: SSA’s governance structure for the review and management of IT  investments is documented in its Capital Planning and Investment Control  guidance. This guidance assigns responsibility for the investment  management process to the agency’s deputy commissioners and other  top-level executives. Further, it describes the policies and processes  guiding management decisions on the selection, control, and evaluation  of all investments. In this regard, the Deputy Commissioner for  Systems/CIO is to ensure that IT is acquired in accordance with the  Capital Planning and Investment Control procedures. Additionally, this  official is responsible for reviewing and obtaining the Commissioner’s  approval of the agency’s annual IT budget and investment plan.
SSA also has a Strategic Information Technology Assessment and  Review (SITAR) board, which is chaired by the Deputy Commissioner for  Systems/CIO and includes the Principal Deputy Commissioner, other  deputy commissioners, and other senior executives responsible for the  business units. These key stakeholders represent the business units that  sponsor the investments and work with the Office of Systems to establish  project requirements and strategic direction; they also help oversee the  investments’ development.
SSA uses its Capital Planning and Investment Control process to manage  its software development projects. According to the agency, the  investment management process is intended to meet the objectives of the  Clinger-Cohen Act by providing a framework for selecting, controlling, and  evaluating investments to help ensure they meet the strategic and  business objectives of the agency.
During the investment selection phase, new projects are to be proposed  by a sponsor—either from a business unit for mission-related projects or  from the Office of Systems—and assigned to a portfolio. Proposals that  identify business needs are to be developed based on the  Commissioner’s priorities or on gap analyses performed by portfolio  teams. The portfolio executives and their teams are to review business- sponsored IT investment proposals and recommend and submit  proposals to the Office of Systems to develop resource estimates on the  proposals. Portfolio executives develop prioritized lists of proposed  projects based on several factors, such as the availability of agency  resources, and prepare recommendations on specific proposals for the  agency IT plan. Next, the prioritized lists are to be combined into the plan  for approval by the SITAR board. The plan is to be comprised of  proposed investments for the next 2 fiscal years, and provide information  on work year requirements. In addition, expected benefits and returns  on investment are to be included for new projects. The SITAR board is  responsible for approving the agency IT plan on an annual basis and is to  modify the plan as needed based on changing priorities. The plan is then  sent to the Commissioner, who provides final approval of the specific  project proposals presented in the plan.
During the control phase, the Office of Systems is responsible for holding  monthly meetings with project managers who are assigned to monitor  investment projects. During these meetings, projects that are not meeting  cost and schedule expectations are to be identified and corrective actions  are to be initiated. One of the objectives of these meetings is to resolve  problems related to underperforming projects without elevating them to  the level of the SITAR board. During the months in which the board  meetings are scheduled, the Deputy Commissioner for Systems/CIO  meets with staff to address any concerns about investments that may be  raised during the meetings. If concerns are raised at a meeting, the  Deputy Commissioner for Systems/CIO is to provide information to the  SITAR board about these investments. In addition, the board is to receive  profiles on the status of each of the agency’s major IT investments. These  profiles should include reports on actual and expended work years, costs,  schedules, and any variances.
During the evaluation phase, the Capital Planning and Investment Control  guide calls for the Deputy Commissioner for Systems/CIO to conduct  post-implementation reviews on projects that have been completed and  deployed for at least 3 months. The purpose of these reviews is to  compare actual project results against planned results in order to assess  performance and identify areas where future decision making can be  improved.
SSA uses two primary oversight systems to monitor the IT projects that  are selected and developed, as well as to store information on unique  projects that are completed. Specifically, these systems are Electronic  General Auditable Documents Store (EGADS) and Prism.
EGADS is the agency’s internally developed system for storing and  accessing final approved versions of project life-cycle documents,  such as IT proposals, among others. Project managers and other  officials, such as subject matter experts, are responsible for providing  updated documents for storage in the system.
Prism is the agency’s web-based project and portfolio management  tool that is intended to track the progress of projects.designed to provide senior management and stakeholders with an  overview of a project’s key performance indicators, such as project  status, schedule, resources, and risks. Project managers are  responsible for managing and updating their project information in  Prism, which is used for reporting during monthly management  meetings.
The agency classifies its IT projects according to five major categories:  (1) development, (2) maintenance, (3) cyclical, (4) planning and analysis  only, and (5) National Computer Center.
Development projects are those that involve the creation of a new  software application, enhancements to an existing application, or  installation of or enhancements to new architecture or hardware.
Maintenance projects pertain to activities required to keep a system  application running. Once a project is completed, SSA places the  system into maintenance mode.
Cyclical projects, such as cost-of-living adjustments, are performed on  a routine basis each fiscal year, and can be done annually, quarterly,  or within some other defined time frame. According to SSA, these  types of projects typically do not include a total redesign or large  enhancement, although some cyclical projects involve smaller  enhancement efforts.
Planning and analysis only projects are undertaken for the specific  purpose of determining if a potential IT solution would be feasible from  a business and technical perspective.
National Computer Center projects are undertaken to keep the center  running, and involve such matters as help desk activities, monitoring  activities, and storage management.
Of these five categories, SSA designates certain ones as Executive  Oversight projects. These projects are the agency’s highest priority  and receive additional senior management oversight.
SSA reported that, from fiscal year 2010 to fiscal year 2012, it devoted  approximately 10,739 work years to the five categories, representing  1,950 actual projects. The resources consisted of SSA personnel, as  well as contractor support that, according to the agency, cost  approximately $600 million for fiscal year 2012. SSA’s projects varied in  size from large major modernization efforts, representing hundreds of  work years annually to small investments of less than 2 work years in  effort. In addition, the time frames for completing the projects varied. For  example, while certain projects were active and spanned over a decade,  others were initiated and completed within a year. For 2010 to 2012, SSA  reported that approximately 70 percent of its projects were active; 25  percent were completed; and 5 percent were withdrawn, on hold, or  approved, but not yet started.
To oversee these projects, SSA managers are to adhere to the Office of  Systems’ Project Management Directive, which identifies numerous project management controls that support key areas of the agency’s  Capital Planning and Investment Control process and are applicable to a  project throughout its life cycle. Specifically, certain control documents  call for critical IT investment information to support the select, control, or  evaluate phase(s). For example, IT proposals support the select phase,  project schedules support the control phase, and lessons learned support  the evaluation phase. Yet other controls, such as risk management,  support all three phases. The project managers store the control  documents in the key management information oversight systems,  EGADS and Prism.
According to the project management directive, the control document can  be required, not required, or conditionally required, depending on the  type of project. For example, a project schedule is required for planning  and analysis only, development, and cyclical projects, but is conditionally  required for a maintenance project and is not required for a National  Computer Center project. Additionally, a cost-benefit analysis is required  for planning and analysis only, development, and National Computer  Center projects, but is not required for cyclical and maintenance projects.  In total, the project management directive identified about 46 types of  control documents for managing those IT projects selected for our  study. Table 1 describes the importance of the 11 types of control  documents that we examined and the kinds of projects for which they  were either required or conditionally required.
The five projects selected for our study included one project from each  project type, with the applicable controls depending in part on the type of  project. Specifically,
Ready Retirement Release 1 was a development project and was  required to develop all 11 types of controls included in our review.The project was an approximately 52 work-year Executive Oversight  project that was part of a larger, broader initiative referred to as Ready  Retirement. The larger initiative is intended to make the retirement  application process more efficient by providing a streamlined web  interface for the public and SSA employees to process claims, thereby  increasing automation and improving service delivery. The Ready  Retirement project is ongoing and has had several releases since its  initial inception.
Internet 3441 was a maintenance project and was required to develop  2 of the 11 control types. The project was an approximately 4.74  work-year effort focused on maintaining a web application used to  gather SSA Form 3441 (i.e., disability report appeal) information from  the public. The project continued until September 2010 when,  according to agency officials, it was combined with another  maintenance project.
Medicare Annual was a cyclical project, which required 6 of the 11  control types. It was an approximately 0.83 work-year project, and  was part of a larger effort to update the Medicare Part D systems.
The goal of the Medicare Annual project was to select a portion of all  subsidy-eligible individuals for an eligibility redetermination. This  redetermination was based on characteristics which were more likely  to have a change in the subsidy amount. For example, a change in  income, resources, and/or household composition can affect an  individual’s eligibility for the subsidy and the amount of the subsidy.
Earnings Case Management System (ECMS) Release 3 was a  planning and analysis only project and required 6 of the 11 control  types. The project was approximately a 0.44 work-year effort that  was initiated and withdrawn in fiscal year 2011. ECMS Release 3 was  part of a larger project intended to support SSA’s Earnings Redesign  initiative and had a goal of transitioning the earnings correction  process from using outdated terminal interface “green screens” to a  more modern intranet application by fiscal year 2015. This particular  project focused on assessing the existing business processes related  to updating the master earnings file.
Help Desk Activities is a National Computer Center project that  required 2 of the 11 control types. The project was initiated in 2002  and from the initiation through fiscal year 2012, it accumulated 800  work years. This project supports SSA’s infrastructure and is within  the Office of Telecommunications and Systems Operations. Examples  of the project’s major functions include providing a call center to  answer users’ questions, providing technology support, ensuring  continued operation of the local area network, and providing hardware  and software support. The Help Desk Activities also include providing  a support team to coordinate and resolve programmatic system  software issues, such as slow response times associated with  disability claims processing systems.
section_title: GAO and Others Have Highlighted the Importance of SSA’s IT Investment Controls
paragraphs: SSA’s Inspector General and we have reported on challenges the agency  has faced in the overseeing its IT investments and the significance of  implementing controls to effectively manage the agency’s projects.
SSA’s Office of the Inspector General reported in September 2006  that, while the agency had generally implemented an earned value  management (EVM) system to manage its major IT projects in  accordance with Office of Management and Budget guidance, some  of the requirements were not based on the agency’s cost-benefit  analyses, which limited the effectiveness of the EVM process and the  agency’s IT development project management. Accordingly, the  Inspector General made recommendations aimed at addressing these  limitations. SSA agreed with the Inspectors General’s  recommendations and has taken steps to address the weaknesses.
We reported on SSA’s IT investment management approach in 2008  and found that it was largely consistent with leading investment  management practices, but it was not applying its investment  management process to all of its investments. Specifically, we noted  the agency was executing a majority of the key practices needed to  build the foundation for managing its IT projects as investments, as  discussed in our IT investment management framework, and it had  made progress in establishing the key practices for managing  investments as a portfolio. However, SSA was not applying its  investment management process to a major portion of its IT budget.  Among other things, we recommended that the CIO develop and  implement policies and procedures to manage IT acquisitions as  investments and manage them using the investment management  framework. SSA disagreed with this recommendation and stated that  its process treated these acquisitions as investments and maintained  them by using an investment management framework, though not the  one described in our framework. However, we noted that these  acquisitions were not subject to the agency’s investment management  select, control, and evaluate processes and were not managed by its  investment board. We further stated that, given that IT products and  services made up the majority of SSA’s IT budget, the investment  board’s involvement was essential to helping ensure effective  management of acquisitions.
In April 2012 we also reported that SSA had undertaken numerous  modernization efforts, but it lacked effective measurement tools to  determine progress. We noted that, since 2001, the agency had  reported spending about $5 billion on the modernization of its systems  and it had undertaken hundreds of modernization projects each year  from 2001 to 2011. Nonetheless, SSA still had major efforts under  way to transition from its aging systems to a more modernized IT  environment. Further, SSA had not fully established quantifiable  performance measures for all its modernization projects or performed  post-implementation reviews, which we had previously recommended  and which would enable the agency to effectively measure its  progress. We also reported that SSA lacked up-to-date and  comprehensive plans to guide its modernization efforts. Accordingly,  among other things, we recommended that SSA develop  comprehensive metrics to effectively gauge modernization progress.  SSA neither agreed nor disagreed with the recommendations, but  described steps it was taking that would address elements of the  recommendations related to IT oversight and updated its  comprehensive Information Resources Management (IRM) plan in  2012.
Further, in October 2013 SSA’s Office of Inspector General reported  that it could not determine whether SSA had realized the planned cost  savings for its IT initiatives because the agency had not calculated  actual savings after project implementation.that SSA did not have a process to assess the overall effectiveness of  its IT capital planning and investment control practices. As a result,  the agency did not know whether its IT investments achieved the  planned savings or any productivity improvements. The Inspector  General recommended that SSA continue implementing a cost- effective post-implementation review process to verify whether its IT  investments are meeting planned savings. SSA agreed with the Office  of Inspector General’s recommendation and said the agency, in July  2013, had developed the framework for post-implementation reviews  to assess IT project performance, and will continue to improve the  process to verify that its investments are meeting planned savings.
Most recently, in a December 2013 report that discussed agencies’  reporting on the IT Dashboard, we noted that, according to SSA  officials, the agency creates new milestones and corresponding  baselines for IT investments at the start of every fiscal year that are  based on annual funding amounts received from SSA’s SITAR  process. This process, known as rebaselining, erases past cost and  schedule variances, prevents the agency from monitoring year-to-year  investment performance, and makes it difficult to estimate and  understand life-cycle costs. We stressed that while this rebaselining  only affected one investment in that review, the process had the  potential to impact SSA’s entire IT investment portfolio. We  recommended that SSA revise its investment management processes  to ensure consistency with the baselining practices identified in our  guidance on the management of capital investments. SSA agreed  with the recommendation and noted that it would take steps to revise  its process.
section_title: Selected IT Projects Did Not Fully Adhere to SSA’s Investment Management Controls or Demonstrate Improved Services
paragraphs: The five IT projects that we reviewed did not fully implement project  management controls consistent with SSA guidance and industry  practices. While SSA had developed the majority of required control  documents, only six fully reflected the content identified in SSA guidance  as essential to effective project management. Most of the control  documents developed had limitations, such as a lack of traceability (which  is needed to trace and track the history of projects and demonstrate that  the projects met requirements) and inaccurate or incomplete information.  The limitations could be attributed, in part, to the use of oversight systems  that did not include all needed data or have the capability to fully support  traceability and a quality assurance function that was limited in scope  and, thus, inadequate to assess control documents effectively.
Further, while SSA indicated that its projects have resulted in  improvements to services, it was unable to demonstrate how the selected  projects had contributed such improvements.
subsections: section_title: Selected IT Projects Had Mixed Results in Applying Required Management Controls
paragraphs: As discussed earlier, to manage and oversee its IT projects, SSA  managers and project teams are to adhere to the Office of Systems’  Project Management Directive to ensure that quality products that satisfy  customer requirements are delivered on time and within budget. In  addition, disciplined project management practices call for the  development of project details such as objectives, scope of work,  schedules, costs, and requirements against which projects can be  managed and executed. This can be facilitated by developing work  products that document these details to ensure the projects are being  managed according to specified practices. Toward this end, the SSA  management directive requires project teams to develop essential  documentation (depending on project type) to support the execution and  management of projects.
However, for the five projects in our review, SSA did not consistently  apply its project management guidance. Specifically, the five projects  developed most of the required control documents. However, while six of  these were developed without limitations, the majority had limitations,  reducing assurance that the projects were being managed in accordance  with disciplined practices. Table 2 shows, for each project, whether  required control documents were developed for each project and had  limitations.
The most common limitations that we noted in the project control  documentation were the lack of full traceability or inaccurate or  incomplete information. A more detailed discussion of each of the five  projects and the extent to which they applied required management  controls follows.
subsections: section_title: Ready Retirement Release 1
paragraphs: As directed by SSA’s guidance, 11 specific control documents were  required for this project, and the project developed all 11. Of these, 5  were developed without identified limitations, including the IT proposal,  cost-benefit analysis, business process description, project scope  agreement, and lessons learned. These controls helped guide the project  by providing critical information on the project’s strategic objectives,  expected benefits and risks supporting the agency’s SITAR process, and  helped ensure that the requirements met user needs. Further, lessons  learned for Ready Retirement Release 1 identified best practices and  areas for improvement which could be used as input for the success of  future projects.
However, the other six control documents had limitations. Specifically,  while project team members produced documentation that identified  certain risks to the project, the documentation did not include mitigation  plans, which are essential for avoiding, reducing, and controlling the  probability of the occurrence of identified risks. Office of Systems officials  stated that the mitigation plans were not available because these  documents were stored in the agency’s previous oversight system,  VISOR, which has been replaced with a more current system, Prism. The  officials explained that documents in the previous system are no longer  accessible.
In addition, the project schedule was not available in the project oversight  system, limiting access to critical information, such as milestones, tasks,  and progress through the life cycle—information that is needed for project  oversight. Further, service requests and system release certifications,  which document the software requirements to be developed and identify  that they have been fully tested and approved, were not traceable to the  project release. According to CMMI, project requirements and related  documentation should be traceable to each other; this is essential for  tracking project history and demonstrating that projects meet  requirements. Thus the project lacked sufficient documentation to allow  oversight officials to independently determine that developed functionality  met established requirements. Further, the function point estimate, which  is used to estimate project size, complexity, and effort, was not used to  manage the project. Finally, the project’s quality assurance was not  effectively implemented in that it did not evaluate the effectiveness of  required management controls, to ensure that all findings and compliance  issues were properly recorded for the project.
section_title: Internet 3441
paragraphs: For this project, two control documents were required—service requests  and system release certifications—and both were developed. However,  both had limitations. Specifically, the service requests were not traceable  to project releases and did not account for all project releases, which, as  called for by CMMI and internal control standards, is necessary for  ensuring that system changes are accurately documented. In addition,  while the system release certification noted that defects had been  revealed during software testing, the certification did not identify these  problems and their resolution, if any, to help demonstrate that the  software was acceptable for release into production.
section_title: Medicare Annual
paragraphs: Of the six control documents required for this project, four were  developed. However, the project could not produce evidence that it had  developed a risk management document. According to Office of Systems  officials, this documentation was not readily accessible because it was  stored in the previous oversight system, VISOR. In addition, the agency  could not provide evidence that a project schedule had been developed.  While the project manager provided a schedule related to the larger effort  this project is associated with, it was at too high a level and not detailed  enough to show how it related to the project.
Of the documentation that was developed, the project scope agreement  did not have limitations. The agreement identified the business goals,  user goals, and requirements applicable to the project. By doing so, SSA  established defined project boundaries that reflected a mutual agreement  between the systems project manager and the business project manager,  which is critical to establishing an understanding between all involved  parties and providing a shared vision for the completed project.
However, three other documents that were developed had deficiencies  that limited traceability between requirements and functionality  developed, as called for by CMMI and internal control standards.  Specifically, the description of the relevant business processes was not  traceable to the project or the specific changes to functionality it was to  implement, and changes to the business processes were not clearly  linked to the scope and objectives of the project. In addition, service  requests associated with the project were not accurately linked to work  occurring in the relevant year, or were not accurately linked to the project.  Finally, the system release certifications lacked traceability to the service  requests and were not properly linked to the appropriate year’s work.  These inaccuracies and inconsistencies made it unclear what work was  actually performed for which project.
section_title: ECMS Release 3
paragraphs: This project developed only three of the six required control documents.  In this regard, it could not demonstrate that it had developed a risk  management document, which project officials stated was stored in the  previous project management system, VISOR. In addition, project officials  said they decided not to pursue the development of the cost-benefit  analysis and project scope agreement for ECMS Release 3 because they  viewed the project as completed after the delivery of the business  process description. However, according to SSA’s project management  guidance, the development of these control documents is required, and  other documentation for this particular project showed that the project  was withdrawn and control documentation that was developed did not  substantiate the completion of the project as claimed by the project  manager. Without control documents defining the project’s scope or a  cost-benefit analyses, it is unclear if the project was cost effective and the  status of the project—if the project was withdrawn or completed as  originally proposed—is uncertain.
In addition, limitations were apparent in the three control documents that  did exist. For example, the scope described in the IT proposal, which  according to SSA’s guidance is to support management decisions to  approve and select a project, was not specific to the outcome of this  planning and analysis project. Rather, the scope described in the  proposal was for the larger development effort of which this project was a  part and which is to be completed by fiscal year 2015. Not defining the  specific project scope can hinder the ability to set clear expectations and  measure benefits. Further, the project schedule was inaccurate and  incomplete in that it was not updated, showed tasks as complete which  had not been completed, and was not clearly aligned with the scope of  the project described in other documentation. Finally, the business  process description, which is to describe changes to a business process  required by the new IT investment, was not traceable to other key  documentation, such as the project scope description (which had not  been developed). In the absence of these critical pieces of information, it  was unclear how needed business process changes were related to  specific project requirements.
section_title: Help Desk Activities
paragraphs: This project was required to develop two control documents and included  evidence that it had done so. Nonetheless, both had limitations. In  particular, the IT proposal did not present the expected performance of  the project and, according to the project managers, contained a $2 million  error in estimated costs. Thus, the project lacked needed information for  making investment decisions. In addition, costs and benefits for this  project were included in a higher-level cost-benefit analysis for SSA’s  data center investment, instead of being identified separately. This meant  that SSA had less assurance that actual costs and benefits for this project  were reflected in investment decisions.
section_title: Limitations in Oversight Systems and Quality Assurance Contributed to Inconsistent Implementation of Management Controls
paragraphs: The inconsistent development of required control documents for the five  selected projects can be attributed in part to limitations in the systems  SSA uses to oversee and manage its IT projects and to a quality  assurance process that was limited in scope.
In several cases, certain management control documents were not  available and Office of Systems officials said this was because these  documents were stored in the agency’s previous oversight system,  VISOR, which contains historical project data and is no longer  operational. The agency’s replacement system, Prism, contains  information only on projects started in fiscal year 2012 or after, and  officials stated they did not see the value of transferring historical data to  the new system. However, in cases where projects are part of larger,  ongoing initiatives, having continuing access to control documents that  reflect the history of the investment or related projects can help better  inform management and facilitate oversight of these related initiatives.  For example, Ready Retirement 1 is part of a broader effort consisting of  multiple, interrelated components. Without access to historical status  reports and project work products containing information on resources,  schedules, and risks, SSA limits its ability to use actual project data to  improve project management and increases its need to rely on project  managers’ institutional knowledge.
In addition, the lack of full traceability among project control documents  existed in part because certain oversight systems, such as EGADS and  its related system for processing service requests, were not originally  designed to distinguish and fully trace project control documents at detail  levels. Traceability is part of requirements management that is  fundamental to having a controlled and disciplined systems engineering  process.Management Analysis Staff within the Office of the Deputy Commissioner  for Systems told us that, in August 2013, the agency had begun planning  for a redesign of its older applications, including EGADS and its service  request processing system, to provide the ability to associate control  documents within and among related projects, among other things. Office   In discussing this matter, the Director of Planning and  of Systems officials said that the projected completion date for this  redesign is June 2014. Further, according to the project scope agreement  for this effort, the redesign is expected to provide a more structured  workflow, eliminate the redundancy of data by leveraging data from other  systems such as Prism, and automate several of the manual processes.  Redesigning these systems to support full traceability should facilitate a  more disciplined system development process. Given that the agency  manages hundreds of projects a year and that SSA officials have stated  that project managers have the ability to combine, split up, and/or rename  projects on a yearly basis, full traceability, supported by readily available  and complete project information, is essential.
Other weaknesses in the implementation of management controls, such  as incomplete and inaccurate information in required documents and  work products, can be attributed in part to weaknesses in SSA’s quality  assurance process. According to best practices such as SEI’s CMMI,  process and product quality assurance should evaluate work products  against applicable process descriptions, standards, and procedures.  Further, federal internal control standards call for evaluations of  management controls, such as quality assurance reviews, to focus on the  effectiveness of controls’ implementation, and this monitoring activity  should ensure that findings are promptly resolved and reported to  management.
While the Office of Systems has a quality assurance process to ensure  projects are following the procedures defined by the office, the quality  assurance reviews that were conducted were limited in their scope and  did not evaluate how effectively the controls were implemented.  Specifically, reviews did not evaluate all project types and were limited to  projects over a certain size. For example, the Office of Systems’ guidance  limits reviews to Executive Oversight development projects and Executive  Oversight cyclical projects that are 2 work years and over. According to the quality assurance representatives, in fiscal year 2012 these criteria  limited the reviews performed to approximately 10 percent of the IT  projects that year (approximately 60 out of 600). However, SSA may have  been better positioned to identify and address the types of control  limitations that we identified if its quality assurance process had been  more rigorous and representative of the agency’s investment portfolio,  including major product types of varying sizes.
SSA quality assurance representatives acknowledged that the reviews  need improvement and, in this regard, said that the agency was in the  process of improving its quality assurance review process to expand the  scope and depth of coverage. Specifically, the officials stated that in  September 2012 the agency conducted an internal appraisal of its quality  assurance program which found that stakeholders had an inconsistent  understanding of quality assurance goals, formal quality assurance  training was not available, reviews did not focus enough on issues with  process implementation, findings were not recorded using a method that  allowed for analysis and identification of needed improvements, and  process trends were not communicated to senior management on a  recurring basis. The officials said the agency developed and, in October  2013, began implementing a strategy to address these issues, which was  to include audits of control documents in the agency’s project oversight  systems. The audits also were to include all Executive Oversight projects  and up to four random non-Executive Oversight projects representing  each Office of Systems component and project type, except National  Computer Center projects.
Further, the implementation plan called for the findings of the audit to be  documented and reported to project managers in a quality assurance tool.  In this regard, the Associate Commissioners were to begin receiving  quarterly reports on quality assurance beginning in January 2014.  However, an Office of Systems official subsequently informed us that this  deadline was not met because of the need to address requested changes  to the report. According to an Office of Systems official, the first of these  reports was presented to the Associate Commissioners in late March  2014, and the agency provided us a copy of this report in early April.  According to our review, the report found that each type of review that  SSA conducted—process review, product review, and audit of control  documents—had identified findings such as limitations in project  schedules, the lack of traceability in requirements, risks not being  discussed with stakeholders, and control documents not being stored  appropriately, which were similar to the limitations that we identified.  While the report noted that some of the findings had been closed, others  remained open and the report offered recommendations for addressing  them. According to the Office of Systems official, findings that were not  closed during the first review will be reported in the second quarterly  report.
These steps, if implemented on an ongoing basis, should lead to  improvements in the quality assurance process, and reduce the risk that  the agency’s controls will not be effectively implemented.
section_title: SSA Has Not Demonstrated How the Selected Projects Have Improved Services
paragraphs: Best practices in IT investment management, such as those described in  our investment management framework, emphasize the importance of  monitoring the performance of investments by, among other things,  collecting information such as costs, benefits, schedule, risk  assessments, performance metrics, and system functionality to support  executive decision making. In this regard, establishing a baseline and  specific performance measures that include how IT contributes to  achieving improved program outcomes is essential for periodically  evaluating a project as it progresses and further assessing a project after  its completion. This assessment allows IT projects to be compared with  one another based on selected criteria, such as expected benefits or  improvements, among others, to effectively manage and prioritize the  portfolio of projects. This critical investment information supports  executive decision making that should ensure IT projects are aligned with  the agency’s strategic plan and meet business needs in an effective and  efficient manner.
SSA has stated that its projects have contributed to improved services.
SSA, Information Resources Management (IRM) Strategic Plan FY 2012-2016   (May 2012). of improved services,extent to which the projects actually contributed to improved services.  This is because the performance measures identified were not specific to  the goals of the individual projects, and the projects did not identify  measurable baselines against which to gauge progress.   the agency was not able to demonstrate the
The IT proposal for the Ready Retirement Release 1 project stated  that it would be considered successful if it simplified the claims  application process and implemented consistent business practices  for taking claims. Yet, the proposal did not provide any specific  performance measures that described how these success factors  could be achieved. In addition, the project did not establish a  performance baseline against which to measure improvements. The  agency attributed some benefits to the project, such as an increase of  250,000 Medicare-only claims submitted from 2011 to 2012 and in the  results of its customer satisfaction survey. However, these benefits do  not clearly reflect the stated goals of simplifying the claims application  process and implementing consistent business practices for taking  claims. In particular, the agency did not provide the measures that  were used to determine the scores on the customer satisfaction  survey, nor did it provide evidence that it had specifically measured  results during the time frame when Ready Release 1 was deployed.  As a result, the extent to which this project effectively contributed to  improved services is unclear.
The Earnings Case Management System Release 3 project  established a goal to transition a specific process—the earnings  correction process—to a more modern intranet application. However,  this goal may have been more appropriate for measuring the overall  Earnings Case Management System project development rather than  for the planning and analyses only segment of the project. According  to the project manager, the intent of the planning and analyses only  portion of the project was to review existing earnings business  processes to identify information needs that have changed in this area  over the years. While the project manager stated that the planning  and analysis only project was complete and that this segment of the  overall project was successful because a business process  description was created, other agency documentation noted that the  planning and analysis only project was withdrawn. Thus it is uncertain  to what extent this project may have contributed to improved services.
For the Help Desk Activities project, specific performance measures  were not defined in the project proposal and then mapped to other  identified performance goals. SSA did identify other metrics to assess  help desk performance, such as stating that its call response time  should be less than 90 seconds and its abandoned call rate should be  less than 12 percent, and project officials stated that these metrics  allow the agency to assess its overall key goals such as systems  availability. However, these metrics related to overall customer  service-level agreements and systems availability. Further, project  officials noted that other documentation related to Help Desk Activities  included performance measures, such as systems’ availability being  expected at 99 percent for online applications, but these measures  applied to its overall data center investment, and the documentation  did not describe how previously identified Help Desk Activities metrics  contributed or were related to these higher-level benefits.
While the agency’s most recent Information Resources Management plan  does include performance measures for various high-level IT-related  “domains,” without project-specific performance measures linked to  expected benefits, SSA is not positioned to demonstrate the extent to  which projects are contributing to the agency’s business goals, such as a  specific amount a project is expected to increase service performance.  Consequently, SSA is not sufficiently positioned to justify IT investments  that are intended to improve its services to the public. Moreover, without  documented and approved baselines of proposed benefits to effectively  measure against, it is unclear which projects are more efficient in meeting  business needs.
section_title: SSA’s IT Human Capital Program Has Identified Skills and Competencies to Support Certain Needs, but Lacks Adequate Planning for the Future
paragraphs: Key to an agency’s success in modernizing its IT systems is sustaining a  workforce with the necessary knowledge, skills, and abilities to execute a  range of management functions that support the agency’s mission and  goals. Achieving such a workforce is dependent on having effective  human capital management,  which includes assessing current and  future agency skill needs by, for example, analyzing the gaps between  current skills and future needs and developing strategies for filling the  gaps, as well as planning for succession. Taking such steps would be  consistent with activities outlined in human capital management models  that we and the Office of Personnel Management have developed. In  addition, GAO’s Standards for Internal Control in the federal government  stresses that management should consider how best to retain valuable  employees, plan for their eventual succession, and ensure continuity of  needed skills and abilities.
In addressing its IT human capital, the Office of Systems identified certain  critical skills and competencies supporting its workforce needs in its  Information Resources Management (IRM) Strategic Plan for fiscal years  2014 through 2017, and in a series of gap reports that it developed in  2008, 2010, and 2012. For example, the IRM plan provided the  approximate number of IT specialists in the agency’s workforce, and  described the agency’s use of contractors.
Further, the agency’s IT human capital needs were reflected in the 2-year  skills inventory gap reports that the Office of Systems had developed. The  most recent such report was developed in 2012 and covered fiscal years  2013 and 2014. An executive officer in the Office of Systems stated that  the agency prepares gap reports every 2 years and that, during the fiscal  year in which the report is being prepared, supervisors identify the current  skills of staff, including their levels of ability—beginner, intermediate, and  advanced—and project what will be needed in the following 2 years. For  example, each of the reports identified a number of general and specific  skill sets needed, such as COBOL and Java.
In addition, to help address any identified skill gaps, the Office of Systems  prepares a training plan explaining how it intends to fill the gaps. The plan  includes information identifying courses that are to be offered to staff to  develop the needed skills. Beyond providing training, the office also  pointed to its reliance on contractors to fill skill gaps.
Nonetheless, while SSA had addressed its IT human resources in this  manner, it had not taken steps that identified the agency’s needs beyond  fiscal year 2014. While identifying skills and competencies that are clearly  linked to the agency’s mission and longer-term IT goals is essential,  neither of SSA’s key human capital planning documents—the IRM plan  and the agency-wide Strategic Human Capital Plan—provided information  about future IT human resources needs.
The IRM plan for fiscal years 2014 to 2017 identified strategic goals  such as “deliver innovative quality services” and “build a model  workforce to deliver high quality service.” Further, the plan stated that  the agency intends to use knowledge management initiatives,  technology training programs, and recruitment and retention  strategies to mitigate any potential loss of institutional knowledge and  to maintain its IT workforce. It also described several IT “domains,”  their current states, and high-level 2-year and “out-year” plans for  each domain. However, the plan did not identify how human  resource levels and staff development strategies link to and support  longer-term plans such as how many staff will be needed to execute  the future plans identified in its IT domains.
SSA’s most recent agency-wide strategic human capital plan, for  fiscal years 2009 through 2012, also did not identify future needs for  IT resources. Although this plan identified general agency goals, such  as to develop leaders at all levels through comprehensive succession  management and improve employee performance by fostering better  management, it did not specifically address what IT human capital  resources would be needed to support these goals.
In addition, SSA does not have a current succession plan for the  replacement of experienced staff supporting its IT efforts. The agency’s  most recent succession plan was issued in 2006, and we recently  recommended that it take steps to update the plan. This is particularly  important given previous experiences that have impacted the agency’s  workforce since 2006, including a hiring freeze and greater movement  toward online services. For example, the Office of Systems has reported  experiencing significant decreases over recent years in its entry-level  staff. Specifically, from fiscal year 2010 to fiscal year 2013, the number of  staff in lower pay grades (i.e., with less seniority) dropped from 318 to 62.  In contrast, the number of staff in the higher grades steadily increased  from fiscal year 2008 to fiscal year 2013. In addition, since 2008, about 20  percent of the Office of Systems’ IT staff have been eligible to retire  annually, which is a significant portion of the workforce. According to  officials in the Office of Earnings, Enumeration, and Administrative  Systems, retirements and the loss of experienced staff have made it  difficult to allocate needed skills to projects. Without planning for the  replacement of experienced staff with critical skills, the Office of Systems  may lack the resources needed to respond to requests for IT support in  an effective manner.
According to an official in the Office of Human Resources, multiple human  capital plans, including an IT human capital plan, are to be folded into a  single, updated plan—referred to as the Human Capital Operating Plan.  The agency had expected to have this plan by February 2014. However,  an Office of Systems official subsequently stated that the agency’s  deadline for developing the revised, consolidated Human Capital  Operating Plan had changed to June 2014. According to this official, the  new estimated completion date for the Human Capital Operating Plan  should allow more time for appropriate stakeholder input. Nonetheless,  the officials could not provide specific information on how future IT human  capital needs are expected to be identified or addressed in these revised  plans, beyond stating that they would include the agency’s updated  succession plan. Until SSA ensures that its human capital planning and  analysis addresses the specific competencies and skills critical to meeting  its future IT needs, the agency jeopardizes its ability to deliver IT support  that effectively contributes to its systems modernization efforts and  improved services.
section_title: Conclusions
paragraphs: Although SSA had applied management controls to the selected IT  projects in our review, limitations raise concerns about their effectiveness.  Specifically, while the five projects developed most of the project  documentation needed to show adherence to management controls, the  majority had limitations. In particular, missing and inaccurate control  documents call into question whether projects are efficiently and  effectively achieving their objectives and whether investment decisions  are well supported. These limitations can be attributed to key oversight  systems lacking accurate and complete project information, as well as  their systems’ inability to support full traceability, and to limited and  inadequate quality assurance reviews of project documents, although the  agency has recently taken steps to improve its quality assurance process.  In addition, the absence of specific performance measures and baselines  makes it difficult to fully substantiate how effectively and efficiently IT  investments have improved the agency’s services. Given the hundreds of  IT projects SSA manages each year, ensuring that management controls  are consistently and effectively implemented is critical to the efficient use  of agency resources.
Regarding human capital, SSA has identified certain critical skills and  competencies; however, an absence of planning beyond 2014 may  jeopardize the agency’s ability to carry out its modernization efforts. In  particular, the agency has not identified IT-specific human capital needs  to support its future goals and it does not have a current IT succession  plan for replacing experienced staff. While agency officials stated that  they expect to develop a revised plan for addressing IT human capital  needs, milestones for this have recently been pushed back, calling into  question the agency’s ability to effectively plan for the resources needed  to support its IT modernization efforts. The importance of addressing  these issues is highlighted by the agency recognizing the significant  decreases in entry-level staff and its hiring freeze as challenges to  effectively staffing certain investments. Without having human capital  plans and analyses that identify and address the agency’s needs for  critical IT skills and competencies, SSA risks not having the resources  and skills sets necessary to carry out its future IT plans in support of the  agency’s mission and goals.
section_title: Recommendations for Executive Action
paragraphs: To address SSA’s project management and human capital deficiencies  for its IT modernization efforts, we recommend that the Commissioner of  Social Security direct the Deputy Commissioner for Systems/Chief  Information Officer to take the following three actions:
Perform effective oversight to ensure that key management control  documents for ongoing and future projects are developed, complete,  accurate, and readily accessible in oversight systems to better  support management, traceability, and project analysis of IT  investments.
Assess control documents supporting the selection of investments,  such as IT proposals, to ensure that they fully identify specific  performance measures and baselines to gauge project success.
Identify future IT needs, including skills needed to support long-term  goals and priorities, in the agency’s updated human capital operating  plan and associated analysis.
section_title: Agency Comments and Our Evaluation
paragraphs: We received written comments on a draft of this report from SSA, which  are reprinted in appendix III. In its comments, SSA stated that it agreed  with our three recommendations and described steps it plans to address  them. For example, the agency stated that it will continue to improve its  governance process by requiring both process and product reviews of  selected IT projects, improve its work to identify performance measures,  and include a comprehensive assessment of IT human capital needs in  its updated IT Human Capital Plan.
In addition, regarding our first recommendation, SSA agreed that our  findings were accurate; however, it noted that our review covered a small,  non-generalizable sample of IT projects. While we acknowledge that the  results of our review cannot be generalized to the entire set of SSA’s IT  projects, the projects we reviewed did represent the major categories of  the agency’s investments. Further, several of the issues we identified  were attributable to weaknesses in oversight systems, which could affect  multiple projects. Therefore, it is important for SSA to continue to  strengthen the oversight of its IT management and governance process.
SSA also provided technical comments, which we have incorporated as  appropriate.
We are sending copies of this report to the Chairman of the Committee on  Ways and Means, House of Representatives, the Commissioner of the  Social Security Administration, appropriate congressional committees,  and other interested parties. In addition, the report is available at no  charge on the GAO website at http://www.gao.gov.
If you or your staff have any questions on matters discussed in this report,  please contact me at (202) 512-6304 or melvinv@gao.gov. Contact points  for our Offices of Congressional Relations and Public Affairs may be  found on the last page of this report. GAO staff who made key  contributions to this report are listed in appendix IV.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: Our objectives were to (1) assess selected information technology (IT)  investments to determine the extent to which they adhere to the Social  Security Administration’s (SSA) investment management controls and are  improving services; and (2) determine how SSA’s IT human capital  program, including the identification and implementation of critical skills  and competencies, is supporting its current and future modernization  efforts.
To address the first objective, we selected a non-generalizable sample of  five IT projects, which were based on analyses of data files provided by  SSA and selection criteria designed to include a variety of the agency’s IT  investment categories and sizes. The review was designed to include one  of each of the five project types—development, maintenance, cyclical,  planning and analysis only, and National Computer Center—defined by  the agency. Specifically, after the data files were determined to be  sufficiently reliable for selecting projects, we judgmentally selected Ready  Retirement Release 1 because it was the largest development project in  the core services portfolio from fiscal years 2010 to 2012 and was related  to SSA’s major investment, Ready Retirement. The remaining four  projects, Earnings Case Management System (ECMS) Release 3,  Medicare Annual, Internet 3441, and Help Desk Activities were chosen  randomly from the planning and analysis, cyclical, maintenance, and  National Computer Center categories, respectively.
We analyzed industry best practices and guidance used to effectively  control, oversee, and manage IT projects. We also analyzed SSA policies  and guidance to determine key documents associated with the agency’s  IT project management directive and quality assurance processes to  identify the types of management control documents, such as project  proposals, cost-benefit analyses, and project schedules, that SSA relies  on for managing its investments. Among those identified, we chose 11  types of management controls for our evaluation. The 11 types of  documents we selected support key management activities at different  points throughout a project’s life cycles. Further, they were called for in  the agency’s IT project management directive and quality assurance  processes, as well as other external best practices; spanned each phase  of the agency’s project life-cycle process; and were represented in each  randomly selected project. The 11 control types also were intended to  cover key management processes that are undertaken during the select,  control, and evaluative phase(s) and provide critical information regarding  the resources and costs committed as well as the potential benefits or  improvements to service. For each of the five projects, we obtained the  associated control documents from SSA, where available. We then  assessed each of the five projects to determine the controls that were  applicable, based on the project type, and whether they were developed  for the selected projects. We analyzed the documents against the  agency’s guidance and other best practices. Specifically, we compared  the control documents to the agency’s IT project management guidance,  as well as best practices in the Software Engineering Institute’s (SEI)  Capability Maturity Model for Integration (CMMI) for Development  (version 1.3), our IT investment management framework, and our  standards for internal control. The best practices and standards call for  control documentation to, among other things, be traceable, accurate,  consistent, and effectively implemented. We assessed the control  documentation and determined if any limitations, such as inaccuracies or  inconsistencies, in these areas existed.
To determine whether SSA had improved services, we analyzed  information that was identified in project documentation for measuring the  performance of each investment, where available. Specifically, we  analyzed the agency’s project proposals and supporting documentation to  determine if quantifiable performance measures and baselines, called for  in best practices such as our IT investment management framework,  were identified and measureable for each project. We also interviewed  project managers, project team members, as well as other agency  officials, including the Deputy Commissioner for Systems/Chief  Information Officer, to understand the agency’s processes and control  documents.
To address the second objective, we obtained and analyzed SSA human  capital plans and data, including its Information Resources Management  (IRM) Strategic Plan for fiscal years 2012 through 2016 and the most  recent updated plan for fiscal years 2014 through 2017, 2-year IT skills  inventory gap reports, IT staffing data for fiscal years 2008-2014, agency- wide succession plan, and its agency-wide human capital plan. We  compared the agency’s plans and documents to best practices we have  identified in strategic human capital management. We also noted any  human capital issues that were identified in interviews with agency  officials or documentation for the selected investments above.
We assessed the reliability of the data that we used to support the  findings in this report by reviewing relevant IT investment files and  program documentation to substantiate evidence obtained from agency  databases. We analyzed agency database instructions and replicated the  system processes with SSA data files and compared the results with  agency-provided data. We also corroborated documentation on program  processes and projects obtained from SSA through interviews with  agency officials to clarify program processes and documentation. We  determined that the data used in this report are sufficiently reliable. We  have also made appropriate attribution indicating the sources of the data.
We conducted this performance audit from November 2012 to May 2014  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
section_title: Appendix II: Description of Selected Projects and Evaluation of Implementation of Management Controls
paragraphs: The following provides a description of each project and our evaluation of  control documents called for by SSA’s Office of Systems Project  Management Directive and other guidance:
Ready Retirement Release 1. Ready Retirement Release 1, which  was to provide online services for Medicare Only and Month of  Election, was an approximately a 52 work-year Executive Oversight  development project completed in 2010. This project was part of a  larger, broader initiative referred to as Ready Retirement. The larger  initiative was intended to make SSA’s retirement application process  more efficient by bypassing the manual claims processes and  providing a streamlined web interface for the public and SSA  employees to process retirement claims, thereby increasing  automation and improving service delivery.
For this project, all 11 of the control documents we selected from  SSA’s Office of Systems’ Project Management Directive were  relevant, with 9 being required and 2 being conditionally required.  SSA developed all 11 documents; however, 6 had limitations. Table 3  describes the 11 control documents, whether or not they were  developed, and any limitations with the documents.
Internet 3441. The Internet 3441 project is a maintenance type  project. The project was focused on maintaining a web application  that electronically gathered SSA form 3441public, and was a 4.74 work-year project that was completed in fiscal  year 2010. The system development for this project was intended to,  in part, provide the form’s information electronically and online and  was started in October 2002. The maintenance efforts began in  October 2003 and continued until September 2010, when agency   information from the  documentationproject manager, this effort was combined with another similar  maintenance project.  noted it as completed. However, according to the  For this project, two control documents were required by SSA’s  Project Management Directive and three were conditionally required.
SSA developed the two required documents, and both had limitations.  Table 4 describes the two required documents, whether or not they  were developed, and any limitations that we identified.
Medicare Annual project. The Medicare Annual project was a  cyclical project that identified up to 400,000 Medicare beneficiaries to  request information to verify continued eligibility or any changes in  benefits. The project was part of a larger effort to update the  Medicare Part D systems. The Medicare Annual project was a 0.83  work-year project completed in fiscal year 2010.
Six control documents were relevant for the Medicare Annual  project—five required and one conditionally required. SSA  developed four of the six documents but did not develop two.  Moreover, three of the four control documents that were developed  had limitations. Table 5 describes the six documents, and whether or  not they were developed and any limitations with the documents.
ECMS Release 3. ECMS Release 3 is a planning and analysis only  project that was performed by SSA in fiscal year 2011. Planning and  analysis only projects are defined by the agency as projects that entail  planning and analysis activities to determine the feasibility of the  project, from both a business and technical perspective. As such,  according to project officials, this particular release focused on  assessing the existing SSA business processes related to updating  the master earnings file. ECMS Release 3 was a part of a larger  Earnings Redesign initiative intended to improve data integrity by  reconciling earnings posted to SSA’s master earnings file with  earnings reported to the Internal Revenue Service by fiscal year  2015.
For this project, five control documents were required by SSA’s  Project Management Directive, and one was conditionally required.  SSA developed three of the control documents but did not develop the  other three. Further, the three control documents that were developed  had limitations. Table 6 describes the control documents, whether or  not they were developed, and any limitations.
Help Desk Activities. The Help Desk Activities is an active, ongoing  National Computer Center project. As discussed earlier, this type of  project is one of five major SSA categories for IT Investments. In fiscal  year 2012, approximately 100 work years were devoted to the project.  This project supports SSA’s infrastructure and is within SSA’s Office  of Telecommunications and Systems Operations. Among other things,  the project provides a call center to answer users’ questions, ensures  continued operation of the local area network, and provides hardware  and software support. The Help Desk Activities also provide a support  team that coordinates and resolves programmatic software system  issues, such as slow response times associated with systems.
Of the 11 types of control documents we reviewed for this project,  SSA’s Office of Systems’ Project Management Directive identified 3  that were applicable—2 required and 1 conditionally required. Of the  2 required controls, both were developed and had limitations. Table 7  describes the 2 required control documents, whether or not they were  developed, and any limitations they had.
section_title: Appendix III: Comments from the Social Security Administration
section_title: Appendix IV: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Christie Motley, Assistant  Director; Michael Alexander; Neil Doherty; Rebecca Eyler; Torrey Hardee;  David Hong; Lee McCracken; Christy Tyson; and Charles Youman made  key contributions to this report.