Agricultural Conservation: USDA Should Improve Its Methods for Estimating Technical Assistance Costs
Published: 2004-11-30T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The U.S. Department of Agriculture's (USDA) Natural Resources Conservation Service (NRCS), working with state and local partners, provides landowners with technical assistance for multiple programs to plan and implement conservation measures that protect soil, water, and wildlife. For years, the Congress has been seeking detailed cost information on this assistance as it examined USDA budget requests. In part, because NRCS's financial system was not designed for estimating future budgets, in 1998 NRCS began developing additional cost data and a computer model for estimating future technical assistance costs. GAO was asked to (1) review NRCS's technical assistance cost estimates and (2) identify causes of any differences between the estimates and actual costs ultimately reported by NRCS.
section_title: What GAO Found
paragraphs: In 2003, NRCS started testing its computer model by comparing estimates of technical assistance costs for 10 Farm Bill conservation programs, with actual costs reported by NRCS. GAO's analysis of these comparisons shows that NRCS's model made estimates, program-by-program, which varied considerably from the agency's actual costs. For fiscal year 2003, for example, NRCS's model estimated that the technical assistance costs for seven Farm Bill programs would be higher by 9 to 50 percent, than NRCS ultimately incurred. For three other Farm Bill programs, the estimates were lower than the agency incurred by 16 to 60 percent. Most of the estimates fell outside NRCS's goal of estimating to within 10 percent of the agency's actual costs. In addition, for the 10 Farm Bill conservation programs combined, NRCS estimated its technical assistance costs at $295 million for fiscal year 2003, which is about 15 percent more than the $257 million that NRCS incurred. NRCS officials generally agreed with this analysis. GAO identified several reasons for the differences between the cost estimates and the actual costs. First, some of NRCS's technical assistance work was delayed, occurring later than NRCS assumed when it estimated its costs. This contributed to some overestimation by the model, according to NRCS officials. Second, NRCS's estimates include costs incurred by NRCS's partners. Such costs are generally not included in the actual costs reported by NRCS. Third, some data NRCS uses in its model are based on inaccurate assumptions. For example, when developing estimates about the time it takes NRCS staff to perform technical assistance tasks for use in the model, NRCS assumes, among other things, that its staff are fully trained and perform technical assistance work without interruption. These assumptions do not reflect actual workplace conditions and lead to underestimates. NRCS officials said they would reconsider these and other assumptions.

section_title: Letter
paragraphs: The U.S. Department of Agriculture (USDA) conducts various conservation  programs that help protect our nation’s soil, water, and wildlife. In  partnership with state and local agencies, USDA’s Natural Resources  Conservation Service (NRCS) provides technical assistance to farmers,  ranchers, and others to help them take part in these conservation  programs. For example, through the Conservation Reserve Program (CRP),  the largest program for retiring farmland for conservation purposes, USDA  and its partners provide technical and financial assistance to eligible  farmers who contract with USDA to take such conservation actions as  planting native grasses in place of crops to prevent erosion. This technical  assistance generally consists of making on-site visits to identify the  conservation needs of the land, advising landowners on selection, design,  and application of conservation measures, determining landowners’  eligibility to participate in the program, and developing and monitoring  contracts the agency signs with landowners to implement conservation  measures. In fiscal year 2003—the latest year for which data are  available—NRCS reported that it spent $257 million on technical assistance  to implement this and nine other farm-related conservation programs. In  future years, NRCS officials are anticipating growth in its conservation  programs, and that this growth will sharply increase the agency’s technical  assistance costs.
For years, the Congress has been seeking more detailed and accurate  technical assistance cost information to better understand NRCS’s costs  and how program growth might affect the agency’s budget. Although NRCS  has had historical and anecdotal information on its technical assistance  expenditures, its financial system has not contained the detailed technical  assistance cost information by program and contract that is needed for  estimating future program budgets. In response to congressional concerns  about this lack of information about the cost of NRCS technical assistance  work, NRCS began developing a method and the cost data needed for  estimating its future technical assistance costs and budgets in 1998. These  efforts include  a workload analysis process, which provides estimates of the time  required for technical assistance tasks, and   a computer model, called the cost of programs model, which uses time  estimates from the workload analysis and other cost data about NRCS  salaries and expenses, for estimating NRCS’s costs and future budgets.
In 2003, NRCS began using cost estimates from its model to provide USDA  and congressional decision makers with detailed information on the cost of  providing technical assistance. In 2004, NRCS also used its model to  estimate the agency’s budget requirements. When technical assistance  funds exceeded actual costs, the excess funds were converted to and used  as financial assistance to landowners, according to NRCS officials. In  addition, to assess and improve the quality of the estimates made by the  model, NRCS compared the model’s program cost estimates for fiscal years  2002 and 2003 to program costs reported by its information systems for  those fiscal years.
Because of your continuing interest in ensuring the development of  accurate information about the costs of NRCS programs, you asked us to  (1) review the technical assistance cost estimates produced by NRCS’s  model and (2) identify the causes of any differences between the estimated  costs and the actual costs ultimately reported by NRCS.
To review the technical assistance cost estimates produced by NRCS’s  model, we assessed NRCS’s comparisons of its model results with costs  reported by NRCS for 10 Farm Bill conservation programs for fiscal years  2002 and 2003. The 10 conservation programs authorized or reauthorized  by the Farm Security and Rural Investment Act of 2002 (the 2002 Farm Bill)  and discussed in this report are the Agricultural Management Assistance  Program, the Conservation Reserve Program, the Environmental Quality  Incentives Program, the Farm and Ranch Lands Protection Program, the  Ground and Surface Water Conservation Program, Klamath Basin, the  Wetlands Reserve Program, the Wildlife Habitat Incentive Program, the  Conservation Security Program, and the Grassland Reserve Program. The  Ground and Surface Water Conservation Program, Klamath Basin,  Conservation Security Program, and Grassland Reserve Program are new  programs authorized by this legislation.
Concerning one key source of the cost information, NRCS’s financial  system, we noted, among other things, that USDA obtained an unqualified  opinion on its financial management activities in fiscal years 2002 and 2003.  This opinion covers, in part, the salary cost data that NRCS relies on when  reporting its costs. To identify causes of differences between the estimated  costs and the costs reported by NRCS, we reviewed NRCS’s assumptions  and data used in the cost estimates. As part of this effort, we analyzed  NRCS’s model and related documentation to determine whether the model  appeared to be a logical and reasonable method for estimating technical  assistance costs. To determine the reliability of workload data used in the  model, we evaluated NRCS’s instructions for developing them, and  obtained the views of NRCS officials and staff who led the development of  the workload analyses in the 10 states that performed over half of the  agency’s CRP technical assistance and over 40 percent of its Wetlands  Reserve Program (WRP) in fiscal year 2002. We performed our work  between September 2003 and October 2004 in accordance with generally  accepted government auditing standards.
subsections: section_title: Background
paragraphs: In 1994, the Congress established NRCS and gave it jurisdiction over  programs of the former Soil Conservation Service, as well as other USDA  financial or technical assistance programs for natural resource  conservation and rural development. With more than 12,000 employees  nationwide—about three-fourths located in its state offices and 2,500 field  offices—NRCS focuses primarily on private and other nonfederal lands.  NRCS staff regularly work in partnership with state, local, and private  entities, using the same case files and technical assistance tools. NRCS’s  primary partners are state conservation agencies and the approximately  3,000 conservation districts nationwide. These conservation districts are  units of local government organized to support local conservation efforts  with their own programs and staff.
When asked to do so by farmers and others, NRCS staff assess farmers’  needs—a process that generally involves traveling to the site. NRCS staff  work with the landowner to develop a conservation plan that describes the  strategy to be used, the schedule of activities, and estimated costs. In some  instances, plans are revised several times until the landowner selects a final  alternative. If the landowner applies to implement the conservation plan  under a Farm Bill program and the land meets the eligibility criteria and is  accepted, NRCS develops a contract. After a contract is developed and  signed, NRCS staff complete paperwork for payments to the landowner.  NRCS staff assist with installation of practices, for example, by surveying  land, providing practice standards and specifications, and ensuring  contractors have carried out the terms of the contract. In addition, NRCS  continues to document activities throughout the life of the contract, which  may be years or decades. Staff also periodically certify that the participant  is complying with the contract terms, depending on the program  requirements.
In 1998, the Chief of NRCS called for a new agencywide effort to improve  NRCS’s accountability by providing better information and analyses on  how the agency uses its resources and what it achieves with its funds. As  part of these efforts, NRCS has taken steps to estimate its future technical  assistance costs and budgets, as described in figure 1.
NRCS’s model for estimating the cost of programs consists of an Excel  spreadsheet that makes program-by-program calculations using such data  as time per task estimates and salary benefits and support costs, as well as  assumptions about the average length of a contract and the proportion of  work that will be performed during each year of the contract. Figure 2  illustrates the information used in NRCS’s cost of programs model.
subsections: section_title: Time Per Task Estimates
paragraphs: NRCS state offices have overall responsibility for developing the time per  task estimates used in the model. NRCS divided the country into 218 areas  and assigned teams of about 7 to 12 NRCS and partner staff to develop time  per task estimates for the work to be done in each area. According to  NRCS’s process for developing time per task estimates, these teams first  define the typical work in their area—that is, the sizes and types of land  they work with and the conservation practices they plan for this land. For  example, a Texas team determined that one type of typical conservation  work in its area was planting grass cover on a 150-acre dairy farm. Using  this description, the team discussed and agreed on estimates of time  required for each of the 29 tasks associated with performing this work,  such as the time it would take them to design a grass-planting plan.  Generally, after developing the time per task estimates, the team leader  submits them to the state time per task leader to review and input into an  NRCS database. The methods of review the state offices use include  comparing the areas’ estimates with each other. If the state leader  encounters substantial unexplained differences among estimates, he or she  generally contacts the team leader for an explanation and to have the  estimate changed, if appropriate. The state leader then submits final time  per task estimates to NRCS headquarters, which also reviews them. If  questions arise, headquarters may ask the state office to perform additional  review, and make changes, if warranted.
Since the quality of data entered into a model affects the estimates it  produces, NRCS has worked to increase the reliability of its time per task  estimates by developing new estimates on three separate occasions and  asking for staff feedback on the quality of the estimating process. In  addition, NRCS plans to update its time per task estimates again because  the tasks and the time required to perform them have changed, along with  program requirements and policies, over the past 5 years.
section_title: Salary Costs
paragraphs: NRCS’s cost model also includes data on average salary costs, which are  based, in part, on data from NRCS’s time and attendance system.  NRCS  staff enter hours that they work by program, and by the activities  associated with those programs, biweekly in the time and attendance  system. For example, they might record working 20 hours on CRP and 20  hours on WRP. For the 20 CRP hours, they may record 10 hours to  determine land eligibility for multiple landowners and 10 hours working  with landowners on conservation plans. These data are reviewed first by  supervisors and then at the state office level.
section_title: NRCS’s Technical Assistance Cost Estimates Differ from Actual Costs Reported by NRCS
paragraphs: NRCS tested cost estimates from its model for fiscal years 2002 and 2003 by  comparing them with the agency’s actual costs. Our analysis of these  comparisons shows that program-by-program estimates from NRCS’s  model vary considerably from the agency’s actual costs as shown in table 1.   These results do not meet the agency’s goal of achieving a difference of no  more than 10 percent between estimates and reported costs.
For fiscal year 2002, the CRP estimate was closest to the actual costs—it  was 5 percent lower than the actual costs reported. Of the remaining  program estimates for 2002, three were higher than the actual cost data  by 48 percent to 302 percent, and four were lower by 19 percent to 36  percent. Altogether, NRCS estimated that its technical assistance costs  for eight of the Farm Bill conservation programs would be about $254  million, 19 percent higher than its actual costs of about $213 million.  NRCS did not estimate costs for two programs that had not yet been  implemented.
For fiscal year 2003, the estimate for the Agricultural Management  Assistance Program, a program that provides financial assistance to  producers in 15 states to, among other things, construct or improve  irrigation structures and plant trees, was closest to the reported costs— it was 9 percent greater than the actual costs. Of the remaining program  estimates for 2003, six were higher than the actual cost data by 17  percent to 50 percent, and three were lower by 16 percent to 60 percent.  Altogether, NRCS estimated its technical assistance costs for 10 Farm  Bill conservation programs would be about $295 million, about 15  percent higher than its actual costs of about $257 million. This estimate  was 4 percent closer to total actual costs than in fiscal year 2002.
For the three largest programs—the Environmental Quality Incentives  Program, CRP, and WRP—the estimates varied from the actual cost data  somewhat less in fiscal year 2003 than 2002. In fiscal year 2002, the  estimates had a spread of 70 percent—from a 22 percent underestimate  to a 48 percent overestimate. In fiscal year 2003, the estimates had a  spread of 38 percent—from a 16 percent underestimate to a 22 percent  overestimate.
section_title: Differences in Estimated and Actual Costs Have Several Causes
paragraphs: We identified several reasons for the differences between NRCS’s  estimated and actual costs.  First, for fiscal years 2002 and 2003, the model  estimated costs based on the assumption that programs would be fully  funded in the beginning of the fiscal year. This did not happen, however,  because the 2002 Farm Bill was enacted later than expected and because  USDA operated under a continuing resolution for a good portion of fiscal  year 2003. According to NRCS officials, as a result, less technical assistance  work was performed than the estimates reflected.  Second, NRCS’s model  includes costs for work performed by partners’ staff and paid for by the  partner organizations, while the actual cost data generally contains only  the costs for the work of NRCS’s staff. Third, NRCS’s model uses some data  that are based on inaccurate assumptions. This is likely to have contributed  to differences between estimates and actual costs reported by NRCS.
subsections: section_title: Actual Timing of NRCS Work Differed from Timing Assumed When Estimating Costs
paragraphs: In several instances, NRCS performed technical assistance work at  different times than NRCS originally assumed it would when estimating  technical assistance costs. First, NRCS estimates have assumed that full  funding would be available for new contracts at the start of the fiscal year,  but in practice, this has not occurred. For example, in 2002, the Farm Bill  was enacted in May, later than NRCS expected, and OMB apportioned  funds to USDA for implementing the Farm Bill programs in July—about  three quarters of the way through the fiscal year.  Since only 2-1/2 months  of the fiscal year remained, different work—and in some cases, less work—  was performed under the Farm Bill than NRCS had anticipated, according  to NRCS officials. The general 2002 sign-up period for CRP contracts did  not start until August 2002, limiting the amount of work performed on new  CRP contracts. Moreover, in fiscal year 2003, USDA operated under a  continuing resolution until receiving fiscal year 2003 appropriations in  February and an OMB apportionment in March—about half way through  the fiscal year. While NRCS can adjust its assumptions, it is not possible to  eliminate uncertainties and variances related to the timing of funding  approvals that cause differences between the estimated and actual  program costs. NRCS officials said that they have been studying the  model’s estimates and modifying some assumptions about workloads to  improve the estimates.
section_title: NRCS Model Included Costs for Work Performed by Partners
paragraphs: Because NRCS’s partners’ efforts are a relatively important part of overall  technical assistance efforts, NRCS has included its partners’ costs in its  model so that it is in a position to estimate total technical assistance costs  to carry out the programs. For example, according to 1999 NRCS staff  estimates, the most recent available, NRCS’s partners were responsible for  about 17 percent of total CRP costs and 15 percent of total WRP costs that  year. Using those percentages, we calculated that NRCS’s partners might  have added about $15 million to these two programs’ fiscal year 2002  technical assistance costs.
To further illustrate the effects of including partners’ costs, we conducted  two comparisons. First, we compared NRCS’s fiscal year 2002 technical  assistance cost estimates, which include partners’ costs, for two  programs—CRP and WRP—with the costs reported by NRCS. We then  made the same comparison but with partners’ costs excluded. (See table 2.)   The results show that the differences between the estimated and actual  costs increase when partner costs are excluded. For the CRP program, for  example, the difference between the estimated and reported costs  increases from a 5 percent underestimate to a 22 percent underestimate  when partner costs are excluded. For WRP, the difference between the  estimated and reported costs also increases from a 22 percent  underestimate to a 34 percent underestimate when partner costs are  excluded.
However, NRCS officials believe that the differences between their  estimates, which include partners’ costs, and the reported actual costs are  less significant because their partner costs have decreased—they could not  say by how much. This may not be the case, however. An NRCS briefing  document reported in 2000 that NRCS expected its partners to perform  more of its workload than in 1999. Similarly, most of the officials we spoke  with from conservation districts—which are key partner organizations— reported that conservation districts have increased the amount of technical  assistance work they have performed in the past few years. Finally,  partner’s costs should not be included when NRCS reports its costs, and  when it compares its estimated and actual costs. In commenting on a draft  of this report, NRCS officials noted that it is important to include partner  costs when estimating the full costs of the programs. They also stated that  estimated costs are used for budgetary purposes and actual costs are the  only costs charged to the government and used in final reports.
section_title: Time Per Task Data Used in the Model Are Based on Inaccurate Assumptions
paragraphs: NRCS’s estimates of technical assistance costs for 10 Farm Bill  conservation programs are developed, in part, using time per task data that  are based on inaccurate assumptions. First, NRCS’s basic assumption used  for developing its estimates—descriptions of typical technical assistance  work—oversimplifies the situations field staff encounter to the extent that  the resulting estimates do not accurately represent the time it takes staff to  do the work. Second, NRCS’s time per task estimates are based on some  assumptions that do not reflect actual workplace conditions. Finally, some  NRCS staff who provide technical assistance are uncertain about how to  allocate the time they spend traveling to and from field locations among  NRCS’s programs.
subsections: section_title: Descriptions of Typical Work
paragraphs: NRCS uses descriptions of typical work as the basis for its time per task  estimates. NRCS guidance tells teams to develop descriptions of their  typical work, but does not tell them how to do this for areas in which  conditions are diverse. In the absence of more specific guidance, staff have  encountered difficulty with the concept of typical work. According to one  team leader, for example, the guidance does not tell teams how to  determine what is typical when there are significantly varying land sizes,  types of farms, or conservation practices, and several team leaders told us  that such variations made identifying typical work difficult or impossible.  For example, a North Dakota team leader said that the conservation work  in his area varies considerably, ranging from installing pipelines to creating  ponds to building fences. Moreover, a Wisconsin team leader told us that at  least three typical descriptions would be needed to represent the variety of  farms in his area: one each for dairy, vegetable, and cranberry farms. Other  NRCS staff commented that it may not be possible to describe what is  typical for western rangeland, where operations vary from hundreds of  acres to tens of thousands of acres. Finally, another team leader told us that  NRCS’s guidance is vague, and as a result, his team interpreted the term  “typical” to mean average, while another understood it to be the median or  middle value.  NRCS staff raised these concerns in 2000, and NRCS officials  are aware of the difficulties staff have encountered in describing their  typical work. According to these officials, the scope of this estimating  problem is nationwide and resolving this concern is important for making  more accurate estimates. NRCS is considering whether it is possible to  resolve difficulties staff face in describing typical work by enlarging the  number of areas for which estimates are developed or collecting data for  more than one “typical” unit per area, and thereby reducing the extent of  diversity within each. Additional information will be needed to determine  whether such an approach will succeed.
section_title: Assumptions about the Workplace
paragraphs: NRCS has directed its staff to base its time per task estimates on three  assumptions that we believe do not reflect actual workplace conditions.   First, estimating staff are to assume that all NRCS staff are fully trained.  This is not the case, however. About 10 percent of current NRCS staff were  hired between 2001 and 2004. According to one NRCS official, staff need 1  to 1.5 years to become able to independently perform most technical  assistance for CRP and 3 to 5 years for WRP. Because not all staff are fully  trained, assuming that they are is likely to inappropriately lower the time  per task estimates. The second assumption is that staff are not interrupted  during their workday. Under normal conditions, staff regularly experience  interruptions that decrease productivity. Assuming that this is not the case  is also likely to contribute to inappropriately lowering the time per task  estimates. A third assumption, which would likely lead to raising the time  per task estimates, is that NRCS staff completely follow NRCS’s policies,  procedures, and guidance in performing work. Actually, however, staff  sometimes take shortcuts that do not comply with all policies, procedures,  and guidance—thereby completing tasks faster than expected.
In contrast, NRCS’s reported actual costs are based on actual work  conditions. That is, these costs reflect the additional time taken by new and  partially trained staff, the added time caused by interruptions that staff  regularly face, and the timesaving shortcuts that staff sometimes take.  Although we could not determine the precise effects of these assumptions,  some information is available to indicate that they warrant reexamination.  For example, NRCS staff reported that by using shortcuts, their CRP and  WRP work took 24 percent and 31 percent less time, respectively, than the  time they had estimated in their workload analysis. NRCS officials said that  they have adjusted the model to take into account some policy and  procedural changes that reduce the workloads of NRCS staff and added  that they would also reconsider the assumptions that we identified.
section_title: Allocation of Travel Time
paragraphs: NRCS staff also reported some confusion about how to allocate their travel  time. The guidance directs staff to divide travel time, which constitutes an  important portion of field staff work time, among different program  activities when necessary. Staff usually drive to meet with landowners  and view land, often traveling to distant locations and working on several  program activities with several farmers during a single trip. When this  occurs, they must determine how much travel time they should assign to  each program. The guidance states that travel time should be “prorated”  among different program activities but does not explain how to do this.  This lack of guidance results in reporting inconsistencies. For example,  staff often visit several farms on a single trip making it difficult to  determine how to prorate this time among multiple program activities.  NRCS officials said that they are aware of this problem but have not yet  developed a solution.
section_title: Conclusions
paragraphs: While we recognize that only 2 years of comparative cost data are available  and that NRCS has been striving to improve its technical assistance cost  estimates, NRCS’s cost estimates differ enough from actual costs reported  by NRCS to be of concern to those who use these estimates.  NRCS’s  overall technical assistance cost estimate for 10 Farm Bill conservation  programs for fiscal year 2003 is closer to the reported cost than the  estimate was for fiscal year 2002, but too much variation is evident on a  program-by-program basis in both years. Until improvements in NRCS’s  technical assistance cost estimating are demonstrated through tests of the  model’s results, we believe NRCS cost estimates should be used with  caution. Without identifying costs incurred by its partners when assessing  the reasonableness of the estimates made by its model, NRCS cannot  ensure the validity of its cost comparisons. Also, unless NRCS modifies its  assumptions to better reflect actual workplace conditions, its technical  assistance cost estimates will not be as precise as they could be. Finally,  without pilot testing its plans for improving descriptions of typical work or  other changes in data development, NRCS cannot be assured that its  investment in its next nationwide workload analysis will be well spent. As  NRCS improves the quality of its workload analysis, including its time per  task estimates, and the assumptions used in the model, we believe more  accurate technical assistance cost estimates will be developed. Moreover,  when these improvements have been made, NRCS will be in a better  position to evaluate the overall quality of its estimating. Further testing in  the years ahead may well be needed to gain a better understanding of the  causes of variations in the program-by-program cost estimates.
section_title: Recommendations
paragraphs: To improve the accuracy, and therefore the usefulness of NRCS’s program  cost estimating, we are recommending that the Secretary of Agriculture  direct the Chief of NRCS to take the following three actions:  clearly identify nonreimbursable costs incurred by NRCS’s partners  when presenting estimates of NRCS’s costs, ensuring that its model’s  estimates are comparable with actual data;    change the assumptions used for developing time per task data for the  model so that they better reflect actual work conditions; and   pilot test the feasibility of proposed changes in the development of the  time per task data, including changes in development of typical work  descriptions in several diverse areas of the country before proceeding  with another nationwide workload analysis.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to USDA for review and comment. We  received oral comments from NRCS, which are summarized below. We also  received technical comments, which are incorporated in this report as  appropriate. NRCS accepted our findings and said they would develop  actions in response to our recommendations. The agency stated that the  report provides the basis for updating the agency’s workload analysis, more  accurately estimating partner contributions to NRCS’s programs, and  making other necessary adjustments to the assumptions in cost estimates.  NRCS also stated that our report rightly points out that some assumptions  used in estimation were inaccurate, but that portions of the report had an  unnecessarily, negative tone. They noted for example that the estimates  would have been closer to actual costs if funding had been available at the  beginning of fiscal years 2002 and 2003. We agree that earlier funding would  have likely helped close the gap between estimated and actual costs. NRCS  also stated that partner contributions could be excluded from the model’s  estimates, but that NRCS wanted to acknowledge the full cost of programs  using partner costs. We agree that partners’ costs can and should be  excluded from the model’s estimated costs when cost estimates are used  for budgetary purposes or for comparison with actual costs. Lastly, NRCS  commented that we found no problems with the logic of the model. We  disagree. The model’s inclusion of costs that the agency did not incur, such  as partners’ costs, is inappropriate when comparing estimates to NRCS’s  actual cost data.
As agreed with your offices, unless you publicly announce its contents  earlier, we plan no further distribution of this report until 30 days from the  date of this letter. We will then send copies to interested congressional  committees, the Secretary of Agriculture, the Chief of NRCS, and other  interested parties. We will also make copies available to others on request.  In addition, this report will be available at no charge on the GAO Web site  at http://www.gao.gov.
If you or your staff have any questions about this report, please call me at  (202) 512-3841. Key contributors to this report are listed in appendix II.
section_title: Objectives, Scope, and Methodology
paragraphs: Our objectives were to (1) review the technical assistance cost estimates  produced by the model and (2) identify the sources of differences that may  occur between the estimates and NRCS’s reported costs.
To review the estimates, we assessed the differences between the NRCS’s  model results compared with the actual costs reported by NRCS. We  compared NRCS’s fiscal years 2002 and 2003 technical assistance cost  estimates by program with the actual costs reported by NRCS, the only  years for which these two sets of costs were available.  To identify sources  of difference between these costs, we assessed assumptions and data used  in the cost estimates. To assess NRCS’s cost model, we analyzed the model  and related documentation to determine whether the model appeared to be  a reasonable method for estimating program costs. In addition, we checked  whether the model formulas, contained in a MS Excel file, used the  appropriate data, and we reviewed the formulas to ensure that they were  logical. In addition, we replicated the model formulas using the proper data  and ensured that the resulting figures matched those shown in the model.  We interviewed NRCS officials responsible for developing the cost model  to gain an understanding of the model and its development. These officials  were staffed in NRCS’s Budget Planning and Analysis Division, Operations  Management and Oversight Division, and in field offices.
We conducted sensitivity analyses to illustrate the possible importance of  different variables in the model. These sensitivity analyses were conducted  using Monte Carlo simulation, which uses random numbers to measure the  effects of uncertainty on model output—in this case, the technical  assistance cost estimates. Our analysis was based on general assumptions  about the probability distributions characterizing some of the variables in  the model.
Also, we interviewed and requested documentation from 20 officials of  state government agencies and conservation district associations to assess  whether the nonreimbursed contributions of conservation districts  increased or decreased in the past several years. We did so for each of the  10 states with the most CRP and WRP contracts.  The 10 states are Illinois,  Iowa, Kansas, Louisiana, Minnesota, Mississippi, Missouri, New York,  North Dakota, and Wisconsin. Of the 20 officials we contacted, 13  responded to our request and provided some information about partners’  contributions.
To assess NRCS’s means for ensuring the reliability of the data used in the  cost model, we traced the time per task and other data to their respective  sources, which included agency reports and databases. Since information  regarding data sources was not always readily available in writing, we met  with NRCS officials who described and provided the sources of the model’s  data. Once we identified the source, we verified that the data had been  correctly transferred from the source to the model. In addition, we  performed limited reliability tests, primarily tests for omitted entries and  outliers, of the available source data. Furthermore, to obtain NRCS’s field  staff views on the reliability of the time per task estimates and time and  attendance data used in the model, we used a semistructured interview  guide to interview all 10 officials leading estimate development efforts in  the 10 states that had the most CRP and WRP contracts in 2002. The 10  states had over half of NRCS’s total CRP contracts and over 40 percent of  its WRP contracts. Using another semistructured interview guide, we  interviewed 10 randomly selected NRCS team leaders (out of 44 leaders)  who each led a team developing time per task estimates in one area in each  of the 10 states.  However, we could not assess the quality of NRCS’s  reviews of its workload analysis at state offices because NRCS state  officials retained insufficient documentation of the reasons for changes in  data made during their reviews.
To assess the reliability of other data used in NRCS’s model, we reviewed  NRCS’s method for developing overhead cost data and salary cost data,  which relies on the time and attendance system. Overall, we noted that  USDA obtained an unqualified opinion on its financial management  activities in fiscal years 2002 and 2003. This opinion covers, in part, the  salary cost data that NRCS relies on when reporting its costs. In addition,  we reviewed NRCS’s development of overhead cost information that is  based on Office of Management and Budget budget object classifications,  which include such costs as rent, utilities, equipment, and supplies.  In  addition, we reviewed an NRCS draft report about the quality of NRCS time  and attendance data. That 2001 NRCS draft report found that about half of  NRCS field offices had deficiencies in documenting their use of time, but  the report did not provide sufficient detail to reveal the precise extent of  the problems. Since then, NRCS has implemented corrective actions,  according to agency officials, and has verified on a limited basis that  improvement has occurred.
Our issues with the reliability of data are discussed throughout the report.  We performed our work between September 2003 and October 2004 in  accordance with generally accepted government auditing standards.
section_title: GAO Contacts and Staff Acknowledgments
subsections: section_title: GAO Contacts
section_title: Acknowledgments
paragraphs: In addition to the individuals named above, Charles W. Bausell Jr., Nancy  Crothers, Anne E. Dilger, Beverly A. Peterson, Lynne M. Dunlay, and Judy  K. Pagano made key contributions.