USDA Systems Modernization: Management and Oversight Improvements Are Needed
Published: 2011-07-20T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The United States Department of Agriculture's (USDA) Farm Service Agency (FSA) is responsible for administering billions of dollars annually in program benefits to farmers and ranchers. Since 2004, FSA has been planning to modernize its information technology (IT) systems that process these benefits with the Modernize and Innovate the Delivery of Agricultural Systems (MIDAS) program. GAO was asked to determine (1) the scope and status of MIDAS, (2) whether MIDAS has appropriate program management, and (3) whether MIDAS has appropriate executive oversight and governance. To do so, GAO reviewed relevant department guidance and program documents and interviewed USDA officials.
section_title: What GAO Found
paragraphs: FSA plans to modernize the systems supporting its 37 farm programs with MIDAS. The implementation cost estimate is approximately $305 million, with a life cycle cost of approximately $473 million. However, the implementation cost estimate is uncertain because it has not been updated since 2007 and does not include cost elements that have since been identified, such as the selection of a commercial enterprise resource planning product. Following completion of its initial phase of program planning in October 2010, MIDAS entered its second of four phases--proof of concept and system design. However, the schedule for this phase, which was to be completed in October 2011, is now uncertain. While FSA officials report that the proof of concept activities are proceeding as scheduled, they have delayed a requirements review milestone until December 2011 and have not yet set a new date for the design review. As a result, the completion date for the second phase and its impact on subsequent phases is uncertain. FSA officials plan to revisit the cost and schedule estimates after completing requirements definition. FSA's program management approach includes many leading practices, but could be strengthened. For example, prior to the proof of concept and system design phase, plans were in place for organizational change and communication, requirements management, and risk. However, a few practices were either partially addressed or not addressed at all in program plans or in the implementation of those plans. For example, an integrated team has not yet been formed with representatives from IT programs that MIDAS depends on for its success. Moreover, the plans do not explicitly call for, and FSA has not produced, a schedule that reflects dependencies with those programs, and risks are not being regularly tracked as planned. FSA's uneven adoption of leading practices is likely to limit the agency's effectiveness in managing system development, and thus its ability to deliver system capabilities on time and within budget. Executive-level governance for MIDAS has not been clearly defined and does not fully follow department IT investment management guidance. Specifically, oversight and governance has been assigned to several department and agency bodies, but roles and escalation criteria are not clearly defined among them. Department officials reported that department guidance is being followed for monthly status reviews, but not for department-level reviews at key decision points. The lack of clarity and definition for the roles of the governance bodies could result in duplication or voids in program oversight, as well as wasted resources. Moreover, because MIDAS is not being governed according to the department's investment guidance, the department may not be rigorously monitoring and managing the program and its risks, and may not have the information it needs to make timely and appropriate decisions to ensure the success of MIDAS.
section_title: What GAO Recommends
paragraphs: GAO is recommending that USDA update cost and schedule estimates, address management weaknesses in plans and program execution, and clarify the roles and coordination among governance bodies. USDA agreed with GAO's recommendations and described plans to address them.

section_title: Letter
paragraphs: The United States Department of Agriculture’s (USDA) Farm Service  Agency (FSA) is responsible for administering billions of dollars annually  in program benefits to farmers and ranchers. Since 2004, FSA has been  planning a program called Modernize and Innovate the Delivery of  Agricultural Systems (MIDAS) to modernize the information technology  (IT) systems that process these benefits. Goals include replacing aging  computer hardware and revamping complex and duplicative farm  program benefits processing by 2014. The estimated life cycle costs for  the program are expected to approach half a billion dollars.
This report responds to your request that we review the progress of FSA’s  modernization program. Specifically, you asked us to determine (1) the  scope and status of MIDAS, (2) whether MIDAS has appropriate program  management, and (3) whether MIDAS has appropriate executive  oversight and governance.
To describe the program’s scope and status, we reviewed program  documents and interviewed agency officials to identify the farm programs,  interfaces, and systems; the milestones and products planned and  delivered; and the costs budgeted and expended. To assess whether  there is appropriate program management, we compared program plans  and artifacts with leading practices for program planning and monitoring,  requirements management, contract management, and risk management  to determine the extent to which practices were planned and executed.  To assess whether there is appropriate executive oversight and  governance, we compared USDA and FSA policies, plans, and artifacts  for MIDAS oversight and governance with our guidance and that of the  Office of Management and Budget (OMB) to determine whether this  guidance has been applied.
We conducted this performance audit from October 2010 to July 2011 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives. See appendix I for a complete  description of our objectives, scope, and methodology.
subsections: section_title: Background
paragraphs: USDA manages and administers benefits programs that support farm and  ranch production, natural resources and environmental conservation, and  rural development. FSA is one of three USDA service center agencies  that manages and administers these benefits to farmers and ranchers.  FSA has three core program areas: farm programs, farm loan programs,  and commodity operations.
The largest of the program areas—farm programs—pays billions of  dollars annually to approximately 2 million farmers and ranchers. As of  November 2008, FSA reported that these five farm programs accounted  for 95 percent of FSA’s budget and transactions.    Direct and Counter-Cyclical Payments Program: offsets losses for a  drop in the market price for a specific crop.   Marketing Assistance Loan Program: provides interim financing to  meet cash flow needs when market prices for commodities are at  harvest time lows.   Noninsured Crop Disaster Assistance Program: provides aid for  uninsured crops that are destroyed through natural disasters.   Crop Disaster Program: provides benefits for crop production or  quality losses during the crop year.   Conservation Reserve Program: provides incentive payments and  cost sharing for projects to reduce erosion, protect streams and rivers,  enhance wildlife habitats, and improve air quality.
FSA administers these programs primarily at its approximately 2,300 local  offices using a variety of computing environments and software  applications to process farm program data, including    a central “Web farm,” consisting of an array of interconnected  computer servers that exchange data in support of data storage and  Web-based applications;   a central IBM mainframe that hosts non-Web applications and data;    a distributed network of IBM Application System 400 computers and a  common computing environment of personal and server computers  at each local office.
We, FSA, and others have reported challenges with the current systems  used to deliver benefits. Specifically, FSA’s information systems    date to the 1980s and are obsolete and difficult to maintain. The  maintenance contract on a key component—the Application System  400 computer—expires in 2013, and FSA anticipates that the contract  will be difficult to renew.   provide farmers and ranchers with limited access to farm programs  through the Internet, so they must primarily visit a local office to  conduct transactions.   are not interoperable. FSA personnel at the local offices must switch  between applications hosted on each system. In addition, the  Application System 400 computers can only store customer  information at a local office. Therefore, customers cannot use different  offices to complete their transactions.   do not satisfy federal directives for internal controls and security.    are difficult to modify or change, hampering FSA’s ability to promptly  implement new benefits programs.
subsections: section_title: Goals and History of MIDAS
paragraphs: In early 2004, FSA began planning the MIDAS program to streamline and  automate farm program processes and to replace obsolete hardware and  software. FSA identified these goals for the program:    Replace aging hardware: Replace the Application System 400  computers with a hosting infrastructure to meet business needs,  internal controls, and security requirements.   Reengineer business processes: Streamline outmoded work  processes by employing common functions across farm programs.  For example, determining benefits eligibility might be redesigned  (using business process reengineering) as a structured series of work  steps that would remain consistent regardless of the benefits  requested.
Improve data management: Make data more readily available to FSA  personnel and farmers and ranchers—including self-service  capabilities—and increase data accuracy and security.
Improve interoperability with other USDA and FSA systems: Integrate  with other USDA and FSA modernization initiatives, including the  Financial Management Modernization Initiative for core financial  services that meet federal accounting and systems standards, the  Geospatial Information Systems to obtain farm imagery and mapping  information, and the Enterprise Data Warehouse to provide enterprise  reporting.
FSA drafted initial requirements for MIDAS in January 2004. It halted  requirements development when program officials decided that the  proposed customized solution would not meet future business needs.  FSA subsequently changed its approach in the summer of 2006 from  customized software to commercial off-the-shelf enterprise resource  planning software.
In February 2008, FSA analyzed how its farm program functions would  map to functions available in an off-the-shelf enterprise resource planning  software suite from vendor SAP, which was selected for two other USDA  modernization initiatives—the Financial Management Modernization  Initiative and the Web Based Supply Chain Management program. This  analysis concluded that MIDAS processes generally mapped to the SAP  software. Based on that analysis and a software alternatives analysis,  FSA decided to proceed with SAP Enterprise Resource Planning as the  solution for MIDAS. FSA also decided to accelerate the time frame for  implementing the solution from the 10 years originally planned to 2 years  for its 2008 business case. To accomplish this, FSA would compress the  requirements analysis phase from 4 years to 5 months, and reduce the  analysis and design phase from 3½ years to 9 months.
In preparation for issuing a request for quotation and selecting a  contractor to define, design, and implement MIDAS with the SAP software  suite, FSA staff visited local offices to document farm program business  processes and to determine requirements for the new system. The  request for quotation for the MIDAS system integrator contract was  released in July 2009; a contract based on this request was awarded to  SRA International in December 2009. The contract start was delayed due  to a bid protest, which was resolved in February 2010, and SRA  International began work in May 2010. By this point, FSA had also  awarded six other contracts for services to support additional aspects of  this initiative, including software licenses, project management support,  and technical support.
FSA hired a MIDAS executive program manager in September 2007 and  drafted a staffing plan in April 2009 that called for 35 to 40 full-time  government employees to oversee the program and its supporting  contracts. The program office reports to the FSA Chief Information Officer  (CIO) and has three functional areas: requirements and project  management, IT solutions, and change management and  communications. The USDA CIO is responsible for MIDAS investment  guidance and direction.
Figure 1 depicts a timeline of key milestones for MIDAS from its inception  through the initiation of work by the system integrator.
In view of congressional concern about the complexity, scale, and  challenges of FSA’s IT modernization, USDA has been required to report  to the committees on Agriculture and Appropriations of the Senate and  House of Representatives on key aspects of MIDAS management,  including cost, schedule and milestones, oversight and investment  management, and integration with other modernization initiatives. In  response, USDA has submitted a series of reports to Congress that  reflect the department’s approach toward the modernization program and  its progress.
section_title: Prior GAO Review Found That Program Cost and Schedule Estimates Were Inadequate
paragraphs: In May 2008, at the request of the House and Senate Committees on  Appropriations, we reported that MIDAS was in the planning phase and  that FSA had begun gathering information and analyzing products to  integrate its existing systems. We determined that the agency had not  adequately assessed the program’s cost estimate, in that the estimate  had been based on an unrelated USDA IT investment. Moreover, the  agency had not adequately assessed its schedule estimate because  business requirements had not been considered when FSA reduced the  implementation time frame from 10 years to 2 years. As a result, we said  that it was uncertain whether the department could deliver the program  within the cost and schedule time frames it had proposed and  recommended that FSA establish effective and reliable cost estimates  using industry leading practices and establish a realistic and reliable  implementation schedule that was based on complete business  requirements. The department generally agreed with our  recommendations.
section_title: Leading Practices for IT Modernization Management
paragraphs: Effective planning and management practices are essential for the  success of large, complex IT modernization efforts. Our reviews of these  practices and experience with federal agencies have shown that such  practices can significantly increase the likelihood of delivering promised  system capabilities on time and within budget. Organizations such as the  Software Engineering Institute at Carnegie Mellon University have issued  guidance on effective planning and management practices for  developing and acquiring software-based systems. These practices  include:    Project planning and monitoring: Project planning establishes a  framework for managing the project by defining project activities and  their estimated cost and schedule, among other things. Project  monitoring provides an understanding of the project’s progress, so  that appropriate corrective actions can be taken if performance  deviates from plans. Effective planning and monitoring employ a  range of resources and tools that promote coordination of and insight  into the project’s activities, such as an integrated project schedule,  which identifies a project’s dependencies with other projects to  facilitate coordination of their tasks and resources.   Requirements management: Requirements establish what the  system is to do, how well it is to do it, and how it is to interact with  other systems. Effective management of requirements involves  assigning responsibility for them, tracking them, and controlling  requirements changes over the course of the project. It also ensures  that requirements are validated against user needs and that each  requirement traces back to the business need and forward to its  design and testing.   Contract management: Effective contract management ensures that  contractor activities are performed in accordance with contractual  requirements and that the acquiring organization has sufficient  visibility into the contractor’s performance to identify and respond to  performance shortfalls. It also ensures that the roles of multiple  contractors are clearly defined in a contract management plan, thus  avoiding confusion or duplication of effort in managing the tasks.   Risk management: Risk management is a process for anticipating  problems and taking appropriate steps to mitigate risks and minimize  their impact on project commitments. It involves identifying and  cataloging the risks, categorizing them based on their estimated  impact, prioritizing them, developing risk mitigation strategies, and  tracking progress in executing the strategies.
For projects such as MIDAS, which involve complex and concurrent  activities, it is important that proven practices be implemented early in the  life of the project so that potential problems can be identified and  addressed before they can significantly impact program commitments.
Federal guidance, along with our framework for managing IT  investments and our prior reviews of federal investments also point to  the importance of having executive-level oversight and governance for  the success of large IT investments. Executive attention helps to ensure  that such projects follow sound business practices for planning, acquiring,  and operating the IT system; meet cost, schedule, and performance  goals; and detect and address risks and problems that could impede  progress toward those goals. When multiple oversight boards govern an  investment, it is critical to define the roles and coordination among them  to avoid duplication of effort and to increase the effectiveness of the  oversight. To help institutionalize such oversight, OMB requires capital  planning and investment control processes, including a department-level  board with the authority to commit resources and make decisions for IT  investments. Such boards are to review the investments at key decision  points against standard evaluation factors. OMB also requires annual and  monthly reporting for such investments. Due to its concerns that  investment review boards have not always been effective, OMB recently  identified additional actions agencies should take to strengthen the  boards, including improving the timeliness and accuracy of program data  available to them.
section_title: MIDAS Is Currently Being Defined; Cost and Schedule Estimates Are Uncertain
paragraphs: FSA plans to modernize all the systems that support its 37 farm programs  (listed in app. II) with the MIDAS program. The implementation cost  estimate is approximately $305 million, with a life cycle cost of  approximately $473 million. However, the implementation cost is  uncertain because it has not been updated since 2007 and does not  include key cost elements. MIDAS is in its second of four phases—proof  of concept and system design. However, the schedule for the current  program phase, which was to be completed in October 2011, is uncertain,  and a key milestone, requirements review, is delayed. As a result, the  completion date for the second phase, and its impact on subsequent  phases, is unknown. FSA officials plan to revisit the cost and schedule  estimates after completing requirements definition.
subsections: section_title: Program Scope Is Generally Defined, but Is Not Reflected in Outdated Cost Estimate
paragraphs: As currently defined, the scope of MIDAS includes modernization of  FSA’s systems for all of its 37 farm programs (listed in app. II). The  modernization effort is to address all of the goals of MIDAS: replace aging  hardware; reengineer business processes across all the farm programs;  improve data access, accuracy, and security; and provide interoperability  with the financial management, geospatial, and enterprise data initiatives.  Figure 2 conceptually depicts the proposed systems, components, and  interconnections, in contrast with those currently used to deliver farm  program benefits.
The program’s estimated life cycle cost is approximately $473 million,  with approximately $305 million for program planning, requirements  definition, system design and development, deployment, and program  support through 2014. FSA considers the implementation cost estimate— which was developed in 2007 and is the most current available—to be  preliminary, with a large degree of uncertainty.
FSA officials reported that approximately $66 million has been obligated  for the program from fiscal year 2009 to June 2011, $61 million of which  has been obligated for seven contracts that supported MIDAS during our  review. Approximately $36 million has been obligated for the system  integrator contract, which is to provide planning, development, design,  and deployment. Approximately $25 million has been obligated for the  remaining six contracts, which are to provide project management  support, development, independent verification and validation, software  licenses, and hosting infrastructure. Table 1 describes these contracts.
FSA officials stated that they have not revised the 2007 cost estimate  because the scope of MIDAS has not changed. However, FSA’s cost  estimate for MIDAS does not reflect costs resulting from program  changes identified since 2007, such as  selection of SAP as the enterprise resource planning software and  mechanism for enterprise reporting;   workshops held with stakeholders in 2010 to identify business    deployment of the financial management initiative and planned  integration with the geospatial and enterprise data initiatives.
In addition, estimated costs have not been included for modernizing  program processes that cannot be supported with the SAP software, or  for implementing any new farm program requirements that may be  enacted in the 2012 farm bill.
In April 2011, FSA officials stated that they would begin revising the  program’s cost estimate in September 2011 and would incorporate new  information gained from requirements development. However, they  could not provide a date for completing the revised estimate because this  information was still being identified.
section_title: Design Milestones Have Slipped; Program Schedule Is Uncertain
paragraphs: MIDAS is to be executed in four phases with incremental deployment of  system capabilities, as recommended by OMB. FSA calls these four  phases planning, proof of concept and system design, initial operating  capability, and full operating capability. These phases were to run from  fiscal year 2010 through fiscal year 2014, as shown in figure 3. FSA  completed the program planning phase in October 2010.
In April 2011, FSA officials reported that the second phase was under  way and that the proof of concept demonstration was on schedule, but  that key milestones for system design would not be completed as  scheduled. FSA officials could not provide new completion dates for the  system design milestones or the second phase. They stated that an  update to the schedule, due in September 2011, would also not be  completed as planned because information needed to revise the schedule  is being identified as the second phase progresses. This uncertainty has  implications for the remaining phases, as discussed in the following  sections.
Project planning. This phase began in May 2010 and was completed in  October 2010—1 month later than planned due to FSA’s requirement that  the system integrator address deficiencies in its planning deliverables.  During this phase, the system integrator developed—and FSA  approved—planning documents that define and detail the management of  processes, products, activities, and milestones for the succeeding phases  of MIDAS, including a project plan, concept of operations, SAP  implementation road map, technical development approach,  organizational change management strategy, and data management  plan. FSA also established a federal program office for MIDAS and filled  most program office positions, including key management positions for  the program director and deputy directors for requirements and project  management, IT solutions, and change management/communications.
Proof of concept and system design. This phase, begun in November  2010, was scheduled to be completed in October 2011. The proof of  concept is to demonstrate several functions of one farm program—the  Marketing Assistance Loan farm program—with an interface to geospatial  systems. This demonstration is to use SAP software in a stand-alone (i.e.,  not production) environment and is to validate certain SAP software  functions. An FSA official stated that the first proof of concept  demonstration was conducted in May 2011 and that field demonstrations  are to be conducted through August 2011.
The system design portion of this phase entails three efforts—defining  requirements, allocating requirements to systems, and designing system  functions. To define requirements, FSA is analyzing the 37 farm programs  to identify the required business processes, including the steps, tasks,  and data currently used for these programs. These processes are also  being re-engineered or optimized by aligning them with nine common  processes where possible. Tasks that do not align with common  processes will be identified as program-specific processes. Both common  and program-specific business processes are to be captured and  baselined as requirements. Technical requirements are to be defined in  conjunction with business requirements and will specify computer  processing power, data storage, network bandwidth, and computer  upgrades to support the processing of MIDAS functions, among other  needs. They are also to address modernization goals, including  consolidation of farm program processing to two existing computing  centers, eliminating the obsolete computers in the local offices; allowing  internal and external access to MIDAS through Web portals; and  integrating MIDAS with the other USDA and FSA modernization  initiatives.
Following requirements definition, FSA plans to conduct an allocation  analysis to determine which business requirements can be supported by  the SAP software. Requirements that cannot be implemented using the  SAP software are to be allocated to the Web farm for implementation. A  high-level design of the MIDAS solution, to include both SAP and Web  farm (non-SAP) system functions, will be based on this requirements  allocation.
In April 2011, FSA officials stated that two key system design  milestones—the system requirements review and the high-level design  review—would not be held as scheduled. According to the December  2010 program schedule, milestones for these events were originally  scheduled for May 2011 and July 2011, respectively. However, FSA  officials do not plan to conduct the system requirements review until  December 2011, and a new date for the high-level design review has not  yet been set because additional information and analysis are needed to  plan this milestone. As a result, the completion date for the second phase  is uncertain.
Initial operating capability. This phase was to be conducted from July  2011 to December 2012––a schedule that has not yet been updated to  reflect delays in the second phase. The initial activities of this phase are  to run concurrently with the proof of concept and system design phase.  Detailed requirements are to be defined for the Marketing Assistance  Loan farm program, including required interfaces, computers, data  storage, and networks. Plans call for augmenting the high-level system  design to reflect these requirements, implementing the design for  modernized Marketing Assistance Loan operations, and deploying it to all  local offices.
Full operating capability. This phase, scheduled from September 2012  to March 2014, is to include detailed requirements definition, design, and  deployment for the 36 remaining farm programs and for farmer and  rancher access to farm program services from their own computers. The  schedule for this phase has also not been updated to reflect delays in the  proof of concept and system design phase.
section_title: MIDAS Plans Reflect Many Leading Management Practices, but Could Be Strengthened
paragraphs: Delivering large IT modernization programs such as MIDAS on time and  within budget presents challenges and risks. Program goals are more  likely to be achieved when managers employ leading practices for  managing program planning and monitoring, requirements, contracts, and  risks. Prior to the proof of concept and system design phase, MIDAS  plans were in place and managers were assigned for these practices.  These plans largely incorporated certain leading practices, although each  management area had at least one practice that was not fully satisfied.
subsections: section_title: Program Planning and Monitoring Are Partially Defined; Implementation Is Incomplete
paragraphs: The success of complex IT modernization initiatives such as MIDAS,  which involve transforming business processes and integrating with other  systems, requires effective program planning and monitoring to ensure  that the intended results are achieved. The Software Engineering  Institute, our work, and recent OMB guidance have identified leading  practices that support effective planning and monitoring to include    assigning a full-time project manager and committed business  sponsor to guide the program;    planning organizational change and communications management to  obtain user acceptance of new ways of doing business;   establishing integrated project teams with external stakeholders and  subject matter experts to facilitate coordination of project activities;   developing integrated project schedules to identify external  dependencies among tasks and resources;   defining earned value management that is compliant with relevant  guidelines to manage contractor and project office development  work; and  tracking and reporting the status of key program milestones—such as  through OMB’s IT investment business case (known as the exhibit  300) and program status reports on OMB’s IT investment Web site  (known as the IT Dashboard).
Of these six practices, FSA has satisfied three, partially satisfied two, and  not satisfied one (see table 2). Specifically, FSA has assigned a program  manager and a business sponsor, has planned and initiated  organizational change and communications management, and planned  for earned value management. However, it has not yet established an  integrated project team that formally commits the support of IT programs  related to the project, developed an integrated project schedule that  specifies related IT program dependencies, or reported clearly on key  MIDAS milestones to accurately convey program progress.
Without a committed integrated project team and an integrated project  schedule that identifies MIDAS dependencies on initiatives outside the  program office, the program may not obtain necessary and timely staff  participation, expertise, and resources, and may not be able to  adequately monitor integration with these initiatives.
Without clear milestone reporting, Congress, OMB, department and  agency management, and other interested parties will have difficulty  tracking the delivery of MIDAS capabilities.
section_title: Requirements Management Is Defined, but User Concerns Need to Be Fully Validated
paragraphs: Defining and implementing disciplined processes for developing and  managing the requirements for a new system can help improve the  likelihood that the system will meet user needs and that it will perform or  function as intended. Leading practices for requirements development  and management include, among other things,    establishing a policy for developing and managing requirements;    assigning and defining the role and responsibilities for a requirements    eliciting and validating user needs;    defining a disciplined change control process; and    ensuring that system requirements are traceable back to business  requirements and forward to detailed requirements, design, and test  cases.
FSA fully satisfied four of these practices, and partially satisfied one (see  table 3). MIDAS requirements and change management plans address all  of these leading practices. However, one practice—the validation of user  needs—was not fully satisfied due to incomplete validation of user needs  (called “pain points”) that had been identified prior to the award of the  system integrator contract.
Unless all the concerns previously expressed by field staff as “pain  points” are systematically validated with respect to MIDAS requirements  and appropriately resolved by the new system or some other means,  MIDAS may not meet user expectations and its acceptance by field staff  may be jeopardized.
section_title: Contract Management Is Defined, but Tasks Could Be Better Delineated among Contractors
paragraphs: Effective project management includes clear definition of authority, duties,  and responsibilities among contractors, and between contractors and  program management. According to the Software Engineering Institute  and our prior work, effective processes to manage and oversee  contracts that support IT projects include    establishing and maintaining a plan for managing and overseeing the  contracts;   assigning responsibility and authority for performing contract  identifying the contract work to be performed and the associated  acceptance criteria; conducting reviews with contractors to ensure cost and schedule  commitments are being met and risks are being managed; and   establishing processes for verifying and accepting contract  deliverables.
FSA fully satisfied four of these practices and partially satisfied one (see  table 4). The system integrator contract and supporting documents  indicate that FSA has planned to use these practices and has applied  them in managing this contractor. In addition, the project management  plan describes the management approach for all the contracts that  support MIDAS, specifies responsibility for overseeing the contracts, and  defines the process for reviewing contractor performance. The plan also  requires that contractor deliverables and acceptance criteria be specified  in the contracts. However, the plan does not clarify contractor roles for  tasks supported by more than one contractor, and does not require that  those roles be delineated in other program or contractor documents.  Table 4 presents a detailed assessment of how FSA has addressed  leading contract management practices.
Unless program plans, schedules, and reports clearly delineate the work  products and activities of individual contractors, program staff,  contractors, and stakeholders may be confused about contractor  responsibilities, which may negatively impact program deliverables or  make it difficult to hold contractors accountable. By eliminating contracts  with the potential for duplicate or confusing efforts, FSA has resolved the  ambiguous roles contained in its plans and can now clearly present the  unique roles of its contractors in updates to its program plans and other  artifacts.
section_title: Risk Management Is Defined and an Inventory Established, but Risks Are Not Regularly Tracked
paragraphs: Risk management is critical in complex IT modernization programs such  as MIDAS to detect and address risks before they adversely impact  project objectives. Leading practices and our prior work recommend    establishing and documenting risk management processes in a risk  management plan from the program’s inception;   assigning a risk manager with the authority to oversee the plan and its    defining a risk inventory, and documenting risks in it, along with  decisions about their priority, probability of occurrence, and impact;  and regularly tracking the status of risks and mitigation efforts and  providing this input to project managers.
FSA satisfied three of these practices and did not satisfy a fourth (see  table 5). Specifically, it has defined its risk management processes in a  risk management plan, designated a risk manager, and established a risk  inventory. However, it has not maintained the risk inventory to track and  report the current status of risks and mitigation efforts to inform MIDAS  managers.
Identifying risks according to the MIDAS risk management plan has  provided FSA managers with an initial understanding of the risks faced by  the program. However, until FSA ensures that its risks have been  consistently identified throughout the course of the program and regularly  updates the status of its risks, it cannot ensure that it is effectively  managing the full set of risks it faces or that progress is being made in  mitigating the risks throughout the life cycle of MIDAS.
section_title: MIDAS Governance Is Not Clearly Defined and Does Not Follow Department Investment Guidance
paragraphs: Oversight and governance of IT investments help to ensure that the  investments meet cost, schedule, and performance goals. When an  investment is governed by multiple boards or bodies, the roles and  coordination among these bodies should be clearly defined, including the  processes and criteria for escalating issues. In addition, we and OMB  recommend that federal agencies establish an executive board, typically  at the department level, to oversee major IT investments. This board  should review investments against criteria at key decision points, such as  investment selection. In addition, OMB requires departmental oversight of  the business cases for major IT investments and monthly status updates  of program cost, schedule, and performance information. Consistent with  federal guidance, USDA requires an executive board to oversee major IT  investments at key decision points and a monthly status review.
Oversight and governance of MIDAS is the responsibility of several  department and agency bodies. Department-level oversight is performed  by the Senior Management Oversight Committee; the Project  Management/Design Decision Committee, which reports to the senior  committee; and a proposed third body called the Modernization Review  Board. In addition, the Modernization Program Management Review  Board operates at the agency level. FSA has not clearly identified this  board’s position in the oversight hierarchy. Table 6 summarizes the  purpose and meeting schedules for these bodies.
However, the roles and coordination of these bodies are not clear in the  following respects:    Certain roles have been assigned to governance bodies without  clear delineation of their scope and criteria for escalating issues.  Charters and plans for the department Project Management/Design  Decision Committee and the agency Modernization Program  Management Review Board describe similar—and potentially  overlapping—roles for overseeing agency IT initiatives. Moreover, the  extent of oversight by the active bodies and criteria for escalating  issues related to cost, schedule, performance, and risk have not been  defined in charters or plans.   A key role has not been assigned. According to the MIDAS risk  register, the Project Management/Design Decision Committee and  Senior Management Oversight Committee are to coordinate  enterprise resource planning among MIDAS and other initiatives, such  as financial management. However, this coordination role has not  been described in charters or plans.   The role of the proposed board has not yet been defined. FSA  officials stated that the USDA Modernization Review Board is to  improve MIDAS governance, but its oversight responsibilities and  processes to do so have not yet been defined.
These concerns have been recognized to some extent by FSA and the  department, but remain unresolved. In October 2010, the FSA  Modernization Program Management Review Board appeared to be  aware of this lack of clarity and recommended that a directory of  governance boards be developed and their respective responsibilities,  decision-making processes, and escalation path be defined. An April  2011 Project Management/Design Decision Committee briefing noted that  the proposed Modernization Review Board would mitigate the risk of  integrating MIDAS with other systems. However, as of May 2011, these  recommended improvements had yet to be provided.
Regarding oversight of MIDAS, none of these boards reviewed MIDAS at  key decision points using criteria defined in department guidance. An  official from the department’s CIO office stated that the Senior  Management Oversight Committee serves as the IT investment executive  board recommended by OMB and required by USDA, although the  committee’s charter and other governance plans do not specify this role.  The committee reviewed MIDAS at the planning gate in October 2010,  but did not use the department’s review criteria. Instead, the review  focused on contract deliverables and did not include project management  office documents such as the MIDAS risk assessment and project  management plan, as called for by department guidance. On the other  hand, department officials reported that MIDAS has complied with  department requirements for business case and monthly status reviews.  A department official reported that USDA’s CIO office has conducted  monthly reviews of MIDAS status and its business case using the  department’s criteria and that the status is posted on the IT Dashboard.  Nevertheless, the dashboard reported in January and March 2011 that  improved oversight is needed for MIDAS.
The lack of clarity and definition for the roles of MIDAS oversight and  governance bodies may result in duplication or voids in program oversight  and wasted resources. Moreover, because MIDAS is not being fully  governed according to department investment guidance, the department  may not be rigorously monitoring and managing the program and its risks,  and may not have the information it needs to make timely and appropriate  decisions to ensure the success of MIDAS.
section_title: Conclusions
paragraphs: After years of planning, USDA is moving forward with its farm program  modernization effort known as MIDAS, which intends to remedy long- standing problems with the supportability, efficiency, and accuracy of  existing systems. The agency has made key decisions regarding the  scope of MIDAS, the contractors that will support system design and  development, and the incremental approach it will use to execute the  program. However, FSA’s implementation cost estimate has yet to reflect  decisions and activities that have occurred since the estimate was  developed in 2007. In addition, key events for the proof of concept and  system design phase, currently under way, have been delayed.  Consequently, agency managers are revising the plans for completing  MIDAS requirements definition, system design, and the cost and  schedule for the program, but are unlikely to finalize these plans until  fiscal year 2012. Given the agency’s prior difficulty with developing  reliable cost and schedule estimates, and our corresponding prior  recommendation, it is critical that FSA and USDA adopt a rigorous and  credible approach for revising estimates and complete them in a timely  manner, so that the department has a basis for effectively managing  program progress and making decisions about needed adjustments.
The challenges USDA is facing in meeting its program commitments are  more likely to be overcome if it can adopt and execute effective  management practices. The management framework established by the  agency in a series of plans reflects many leading practices for program  planning and monitoring, requirements, contracts, and risk. Moreover,  FSA has followed through on these plans to some extent by staffing  government managers in these areas and instituting mechanisms to  promote use of the practices, such as contract provisions for earned  value management. However, MIDAS management could be further  strengthened through improved definition and execution of these and  other leading practices, specifically by chartering and operating an  integrated project team; fully documenting MIDAS dependencies on other  departmental IT initiatives in an integrated project schedule; clearly  identifying and reporting key incremental milestones to OMB; validating  all previously identified user concerns against MIDAS requirements;  clearly delineating contractor roles and responsibilities; and consistently  identifying and regularly tracking and reporting the status of MIDAS risks.  By applying its plans and embracing other proven management practices,  FSA will stand a better chance of surfacing and resolving issues before  they can derail the program.
The agencywide impact of MIDAS and its dependence on other IT  initiatives point to the need for clearly defined and effectively executed  oversight. However, the roles and coordination among oversight bodies  are not clearly defined and USDA’s well-defined investment oversight  guidance is not being fully executed. Providing adequate and efficient  oversight for MIDAS in such an environment presents a challenge that  could be avoided if USDA and FSA delineate governance roles and  responsibilities and execute them accordingly.
section_title: Recommendations for Executive Action
paragraphs: To increase the likelihood that the United States Department of  Agriculture (USDA) will be able to successfully define, develop, and  deploy the Modernize and Innovate the Delivery of Agricultural Systems  (MIDAS) program, we recommend that the Secretary of Agriculture direct  the chief information officers of USDA and the Farm Service Agency  (FSA) to take the following three actions:    To ensure that the department can effectively oversee MIDAS cost,  schedule, and performance commitments, FSA should   develop timely cost estimates for MIDAS’s remaining phases, its  overall development and deployment, and its life cycle, to  incorporate the program changes previously omitted and any  others recently identified and   develop complete and detailed schedules for the program’s  current and remaining phases that take into account the milestone  delays from the program’s second phase and a requirements  baseline.   To ensure that FSA is employing leading practices for program  planning and monitoring, requirements management, contract  management, and risk management for MIDAS, the agency should  charter and operate an integrated project team that commits  stakeholders to the program from other USDA information  technology (IT) initiatives;   establish an integrated project schedule that identifies tasks,  dependencies, and resource commitments and contention  between MIDAS and other department IT initiatives; clearly track key milestones, and report their status in the  program’s business case and on the Office of Management and  Budget’s IT Dashboard; validate all of the 591 user pain points against the requirements  and document the results of this validation, including points that  will not be addressed by MIDAS;   update the program’s management plans to clearly delineate the  roles and responsibilities of contractors assigned to the same  tasks; and   document the status of resolved and unresolved risks initially  identified in November 2010, identify and maintain any unresolved  risks from that period in the current risk register, and regularly  track risks and update the risk register according to the program’s  risk management plan.   To ensure the effectiveness of MIDAS oversight and the efficiency of  its governance bodies, the department and agency should collaborate  to   delineate the roles and responsibilities of the governance bodies  and clarify coordination among them, to include criteria for  escalating issues and   document how the department is meeting its policy for IT  investment management for MIDAS, to include investment  reviews.
section_title: Agency Comments
paragraphs: In written comments on a draft of this report signed by the Administrator,  Farm Service Agency, and reprinted in appendix III, USDA generally  agreed with the content and recommendations and described actions and  time frames to address the recommendations. For example, the  department stated that it will revise MIDAS schedule and cost estimates  for this year’s capital planning submission based on fiscal year 2011  planning, requirements, and design sessions, and will be able to develop  more precise estimates at the completion of primary blueprinting and  design in the first quarter of fiscal year 2012. The department described  improvements to address our other recommendations, including  integration processes with other initiatives; requirements validation; risk  management; and department-level governance, to be completed by the  end of the second quarter of fiscal year 2012.
We are sending copies of this report to interested congressional  committees, the Director of the Office of Management and Budget, the  Secretary of Agriculture, and the Administrator of the Farm Service  Agency. In addition, this report will be available at no charge on the GAO  Web site at http://www.gao.gov.
If you or your staffs have any questions about this report, please contact  me at (202) 512-9286 or at pownerd@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. Key contributors to this report are listed in  appendix IV.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: Our objectives were to determine (1) the scope and status of the  Modernize and Innovate the Delivery of Agricultural Systems (MIDAS)  program; (2) whether MIDAS has appropriate program management; and  (3) whether MIDAS has appropriate executive oversight and governance.
To determine the program’s scope and status, we reviewed planning  documents to identify the farm programs included in MIDAS, the required  interfaces to other United States Department of Agriculture (USDA) and  Farm Service Agency (FSA) modernization initiatives, and the proposed  technical approach to the program. We also reviewed the fiscal year 2012  business case (called the exhibit 300), program schedules, oversight  reviews from October 2010, and a 2010 FSA report to Congress to  identify the active contracts supporting MIDAS and to determine the  program’s phases, due dates, and phase completion status. To determine  whether FSA completed the planning phase as scheduled, we identified  the deviation between the planned and actual completion dates. We also  examined selected products produced during that phase. We identified  the cost estimate and its limitations using these sources and a 2009 third- party report to Congress on FSA modernization. We interviewed USDA  and FSA officials to clarify information in the documents we reviewed and  to more fully understand the program’s progress and status.
To determine whether MIDAS has appropriate program management, we  identified leading management practices for four areas that we and others  have previously found to be important for the success of large information  technology (IT) programs—planning and monitoring, requirements  management, contract management, and risk management. We then  reviewed plans to determine if they addressed these leading practices.  For the four management areas, we examined plans, organization charts,  and program records to determine whether and when managers had  been assigned. To the extent that MIDAS had progressed to a stage  where implementation of these practices would be appropriate, we  reviewed program artifacts and interviewed program officials to determine  the extent to which the practices were in place.
We assessed a practice as being satisfied if the evidence provided by  USDA and FSA officials demonstrated all aspects of the leading practice.  We assessed a practice as being not satisfied if the evidence did not  demonstrate any aspect of the leading practice, or if no evidence was  provided by USDA or FSA for that practice. Finally, we assessed a  practice as being partially satisfied if the evidence demonstrated some,  but not all, aspects of the leading practice. Additional considerations in  our evaluation of each management area follow.    Project planning and monitoring: We compared program plans,  including the project management plan and supporting  documentation, against leading practices to determine whether such  practices were specified in the plans. We also examined program  artifacts and records to determine the extent to which an integrated  project team, an integrated project schedule with external  dependencies, and tracking and reporting of program progress  outside the program were in place. Due to the early stage of the  program, we did not verify whether earned value management had  been executed and reported as planned or whether organizational  change and communications activities had been executed as planned.   Requirements management: We compared the requirements  management plan and related documents against leading practices to  determine whether such practices had been specified in the plans.  Because requirements were in the early stages of being defined  during the period of this review, we did not verify whether FSA was  executing its requirements management approach as planned.  However, we reviewed a 2008 requirements document containing  previously elicited user requirements and interviewed FSA officials to  determine how those requirements had been validated.   Contract management: We compared program plans, including the  project management plan and supporting documentation, against  leading practices to determine whether such practices had been  specified in the plans. Due to the critical role of the system integrator  contract in achieving program goals, we focused our assessment on  this contract, the deliverables specified in this contract, and the review  criteria for one deliverable—the strategy plan to decompose the  requirements document. We verified whether the review criteria had  been applied to this deliverable. We did not verify whether other  planning phase contract deliverables had been evaluated by the gate  review panel according to corresponding review criteria. We  compared the descriptions of contractor tasks from contract  management documents to each other and when we identified similar  or identical tasks for different contractors, we interviewed FSA officials  to obtain their explanations for the roles of each contractor and to  clarify the contract management documentation. We reviewed the risk  inventory to determine whether duplicate contractor roles had been  identified as risks and how the risks were described.   Risk management: We compared the risk management plan and  supporting documentation against leading practices to determine  whether such practices had been specified in plans. We also reviewed  the November 2010 risk inventory to assess whether risks had been  aligned with risk factors such as mitigation plans and status. We did  not assess whether FSA assigned the risk indicators according to the  criteria in the risk management plan. To assess whether the risk  inventory was being updated, we compared the November 2010 risk  inventory to risk inventories from January 2011 and May 2011 to  characterize overall changes to risks, mitigation strategies, and status,  and to determine whether the inventories clearly captured progress in  addressing selected risks.
To determine whether MIDAS has appropriate executive oversight and  governance, we reviewed USDA guidance for investment management,  project plans, charters, and meeting minutes for the governance bodies,  agency presentations, and 2009 and 2010 USDA and FSA reports to  Congress to identify the executive oversight and governance bodies,  responsibilities, and hierarchy for MIDAS. We also interviewed USDA and  FSA officials about MIDAS governance structure and practices. We  compared the information we obtained with USDA’s capital planning and  investment control guidance, which comports with federal IT investment  management guidance, and with our IT investment management  framework to ascertain whether USDA had complied with its own  guidance for overseeing the investment and the extent to which  governance bodies, their responsibilities, and processes had been  defined.
We performed our work at the USDA office in Washington, D.C. We  conducted this performance audit from October 2010 to July 2011 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
section_title: Appendix II: MIDAS Farm Programs
section_title: Appendix III: Comments from the Department of Agriculture
section_title: Appendix IV: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact name above, the following staff made key  contributions to this report: Paula Moore (assistant director), Neil Doherty,  Claudia Fletcher, Nancy Glover, Javier Irizarry, and Karl Seifert.