Defense Acquisitions: Actions Needed to Ensure Value for Service Contracts
Released: 2009-04-23T00:00:00

Letter

DOD Continues to Face Challenges in Employing Sound Practices When Contracting For and Managing Service Contracts

It is essential that DOD employ sound practices when using contractors to support its missions or operations to ensure the department receives value. This means clearly defining its requirements, using the appropriate contract type, and properly overseeing contract administration. Our work, however, has repeatedly identified problems with the practices DOD uses to acquire services. Further, an overarching issue that impacts DOD’s ability to properly manage its growing acquisition of services is having an adequate workforce with the right skills and capabilities. Collectively, these problems expose DOD to unnecessary risk and make it difficult for the department to ensure that it is getting value for the dollars spent. Since fiscal year 2001, DOD obligations for service contracts have doubled while its acquisition workforce has remained relatively unchanged (see fig. 1).

Properly Defined Requirements are Essential to Obtaining Value

Properly defined requirements—whether at the DOD-wide level or the contract level—are a prerequisite to obtaining value for the department. At the DOD-wide level the department should have an understanding of what it needs to contract for and why. However, we have frequently noted that the department continues to be challenged to understand how reliant it is on contractors and has yet to clearly determine what services it should obtain from contracts and what services should be provided by the military or DOD civilian employees. Furthermore, DOD lacks basic data about its service contracts that could help it determine how it contracts for services and how reliant it is on contractors. For example, at this time, the department does not have complete and accurate information on the number of services contracts in use, the services being provided by those contracts, the number of contractors providing those services, and the number and types of contracts awarded.

Once DOD determines what services contractors should provide, both the contractor and the government need to have a clear sense of what the contractor is required to do under the contract. Poorly defined or changing requirements have contributed to increased costs, as well as services that did not meet the department’s needs. The absence of well-defined requirements and clearly understood objectives complicates efforts to hold DOD and contractors accountable for poor acquisition outcomes. For example: DOD sometimes authorizes contractors to begin work before reaching a final agreement on the contract terms and conditions, including price. These types of contract actions, known as undefinitized contract actions, are used to meet urgent needs or when the scope of the work is not clearly defined. In July 2007, we reported that, DOD contracting officials were more likely to pay costs questioned by Defense Contract Audit Agency (DCAA) auditors if the contractor had incurred these costs before reaching agreement with DOD on the work’s scope and price. In fact, DOD decided to pay the contractor nearly all of the $221 million in questioned costs after making a determination based on additional information. The lack of timely negotiations contributed significantly to DOD’s decision—all 10 task orders were negotiated more than 180 days after the work commenced. The negotiation delays were in part caused by changing requirements, funding challenges, and inadequate contractor proposals.

In both July 2004 and September 2006 we reported that a disagreement between a contractor and DCAA on how to bill for services to feed soldiers in Iraq resulted in at least $171 million in questioned costs that DOD did not pay. The disagreement regarded whether the government should be billed on the camp populations specified in the statement of work or on the actual head count. A clearer statement of work, coupled with better DOD oversight of the contract, could have prevented the disagreement and mitigated the government’s risk of paying for more services than needed. Negotiations between the contractor and DOD resulted in a settlement whereby $36 million would not be paid to the contractor.

On the other hand, requirements that provide DOD with a greater level of service or performance than required can undermine the department’s efforts to ensure value. For example: In December 2008, we issued a report on performance based logistics, which is defined by DOD as the purchase of performance outcomes (such as the availability of functioning weapon systems) through long-term support arrangements rather than the purchase of individual elements of support—such as parts, repairs, and engineering support. In that report, we noted for eight of the performance based logistics arrangements we reviewed, the contractors significantly exceeded some of the contractual performance requirements. We further noted that since the government is paying for this excess performance, the performance based logistics arrangement, as structured, may not provide the best value to the government. For example, since 2002, the average annual operational readiness for the Tube-launched, Optically-tracked, Wire-guided missile – Improved Target Acquisition System has not been below 99 percent, and the system’s operational readiness has averaged 100 percent since 2004. According to a program official, the Army’s readiness standard for this system is 90 percent. Despite the Army’s standard, it continued to include a performance incentive that encouraged higher levels of performance when negotiating a follow-on performance based logistics contract in 2007. The performance incentive includes payment of an award fee that encourages operational readiness rates from 91 to 100 percent, with the highest award fee paid for 100 percent average operational readiness.

Selected Contract Type and Business Arrangements Not Always Appropriate

When contracting for services, DOD has a number of choices regarding the contracting arrangements to use. Selecting the appropriate type is important because cost reimbursable contracts may increase the government’s cost risk whereas firm-fixed price arrangements transfer some of that cost risk to the contractor. While use of the appropriate contract type is important, it is not the sole factor in a successful acquisition outcome—as noted in this statement, good requirements and oversight of contractor performance are also important. We have found that DOD did not always use the contracting arrangements that would result in the best value to the government. For example: In January 2008, we that reported the cost-plus-fixed fee provisions of a task order issued by the Army to repair equipment for use in Iraq and Afghanistan required the Army to pay the contractor to fix equipment rejected by Army inspectors for failing to meet the quality standard established in the task order. Under the cost-plus-fixed fee maintenance provisions in the task order, the contractor was reimbursed for all maintenance labor hours incurred, including labor hours associated with maintenance performed after the equipment failed to meet the Army’s maintenance standards. This resulted in additional cost to the government. Our analysis of Army data between May 2005 and May 2007 showed that the contractor worked about 188,000 hours to repair equipment after the first failed Army inspection at an approximate cost to the government of $4.2 million.

In June 2007, we found numerous issues with DOD’s use of time-and- materials contracts. DOD reported that it obligated nearly $10 billion under time-and-materials contracts in fiscal year 2005, acquiring, among other services, professional, administrative, and management support services. Some specific examples of the services DOD acquired included subject matter experts in the intelligence field and systems engineering support. These time-and-materials contracts are appropriate when specific circumstances justify the risks, but our findings indicate that they are often used as a default for a variety of reasons—ease, speed, and flexibility when requirements or funding are uncertain. According to DOD, time-and- materials contracts are considered high risk for the government because they provide no positive profit incentive to the contractor for cost control or labor efficiency and their use is supposed to be limited to cases where no other contract type is suitable. We found, however, that DOD underreported its use of time-and-materials contracts, frequently did not justify why such contracts were the only contract type suitable for the procurement, and inconsistently monitored these contracts.

In 2007, we also reported that DOD needed to improve its management and oversight of undefinitized contract actions (UCAs), under which DOD can authorize contractors to begin work and incur costs before reaching a final agreement on contract terms and conditions, including price. The contractor has little incentive to control costs during this period, creating a potential for wasted taxpayer dollars. DOD’s use of some UCAs could have been avoided with better acquisition planning. In addition, DOD frequently did not definitize the UCAs within the required time frames thereby increasing the cost risk to the government. Further, its contracting officers were not documenting the basis for the profit or fee negotiated, as required. As such, we called on DOD to strengthen management controls and oversight of UCAs to reduce the risk of paying unnecessary costs.

In July 2004, we reported that the Air Force had used the Air Force Contract Augmentation Program contract to supply commodities for its heavy construction squadrons because it did not deploy with enough contracting and finance personnel to buy materials quickly or in large quantities. In many instances, the contractor provided a service for the customer, such as equipment maintenance, in addition to the procurement of the supplies. In other cases, however, the contractor simply bought the supplies and delivered them to the customer. In July 2004 we noted that the contract allowed for an award fee of up to 6 percent for these commodity supply task orders. While contractually permitted, the use of a cost-plus-award-fee contract as a supply contract may not be cost- effective. In these instances, the government reimburses the contractors’ costs and pays an award fee that may be higher than warranted given the contractors’ low level of risk when performing such tasks. Air Force officials recognized that the use of a cost-plus-award-fee contract to buy commodities may not be cost-effective. Under the current contract, commodities may be obtained using firm-fixed-price task orders, cost-plus award-fee task orders, or cost-plus-fixed-fee task orders.

Inadequate Oversight of Contractor Performance

We reported on numerous occasions that DOD did not adequately manage and assess contractor performance to ensure that its business arrangements were properly executed. Managing and assessing post-award performance entails various activities to ensure that the delivery of services meets the terms of the contract and requires adequate surveillance resources, proper incentives, and a capable workforce for overseeing contracting activities. If surveillance is not conducted, is insufficient, or not well documented, DOD is at risk of being unable to identify and correct poor contractor performance in a timely manner. For example: Our 2008 review of six Army services contracts or task orders found that contract oversight was inadequate in three of the contracts we reviewed because of a lack of trained oversight and management personnel. For example, in the contracting office that managed two of the contracts we reviewed, 6 of 18 oversight positions were vacant. One of the vacant positions was the performance evaluation specialist responsible for managing the Army’s quality assurance program for two multi-million dollar contracts and training other quality assurance personnel. Other vacant positions included three contract specialists responsible for, among other tasks, reviewing monthly contractor invoices. As a result of these vacancies, the contracting officer’s representative was reviewing contractor invoices to ensure that expenses charged by the contractor were valid, a responsibility for which he said he was not trained. We also reported that contract oversight personnel for the Army’s linguist contract were unable to judge the performance of the contractor employees because they were generally unable to speak the languages of the contractor employees they were responsible for overseeing.

DOD has, over the last several years, emphasized the use of performance based logistics arrangements, in part, to reduce the cost of supporting weapon systems. However, in December 2008, we reported that although DOD guidance recommends that cost data be captured for performance based logistics contracts to aid in future negotiations, we found program offices generally did not receive detailed cost data and only knew the overall amounts paid for support. For example, for the 21 fixed-price arrangements in our sample, only two program offices obtained contractor support cost data reports. We also reported that, in seven out of eight programs we reviewed where follow-on, fixed-price performance based logistics contracts had been negotiated, expected cost reductions either did not materialize or could not be determined.

In our September 2008 review of services contracts supporting contingency operations, we reported the Army’s oversight of some of the contracts was inadequate in part because contracting offices were not maintaining complete contract files documenting contract administration and oversight actions taken, in accordance with DOD policy and guidance. As a result, incoming contract administration personnel did not know whether the contractors were meeting their contract requirements effectively and efficiently and therefore were limited in their ability to make informed decisions related to award fees, which can run into the millions of dollars.

In December 2006, we reported that DOD did not have sufficient numbers of contract oversight personnel at deployed locations, which limits its ability to obtain reasonable assurance that contractors are meeting contract requirements efficiently and effectively. For example, an Army official acknowledged that the Army struggled to find the capacity and expertise to provide the contracting support needed in Iraq. Similarly, an official with the LOGCAP Program Office told us that the office did not prepare to hire additional budget analysts and legal personnel in anticipation of an increased use of LOGCAP services due to Operation Iraqi Freedom. According to the official, had adequate staffing been in place early, the Army could have realized substantial savings through more effective reviews of the increasing volume of LOGCAP requirements. A Defense Contract Management Agency official responsible for overseeing the LOGCAP contractor’s performance at 27 locations noted that he was unable to visit all of those locations during his 6-month tour to determine the extent to which the contractor was meeting the contract’s requirements.

In December 2005, we reported that DOD, in using award fee contracts, routinely engaged in practices that did not hold contractors accountable for achieving desired acquisition outcomes. These practices included evaluating contractors on award-fee criteria not directly related to key acquisition outcomes; paying contractors a significant portion of the available fee for what award-fee plans describe as “acceptable, average, expected, good, or satisfactory” performance; and giving contractors at least a second opportunity to earn initially unearned or deferred fees. As a result, DOD had paid an estimated $8 billion in award fees on contracts in our study population, regardless of whether acquisition outcomes fell short, met, or exceeded DOD’s expectations. As such, we recommended that DOD improve its use of fees by specifically tying them to acquisition outcomes in all new award- and incentive-fee contracts, maximizing contractors’ motivation to perform, and collecting data to evaluate the effectiveness of fees.

In March 2005, we reported instances of insufficient surveillance on 26 of 90 DOD service contracts we reviewed. In each instance, at least one measure to ensure adequate surveillance did not take place. These measures include (1) training personnel in how to conduct surveillance, (2) assigning personnel at or prior to contract award, (3) holding personnel accountable for their surveillance duties, and (4) performing and documenting surveillance throughout the period of the contract.

DOD has Taken Some Steps to Address Service Contract Management and Oversight Challenges

GAO’s body of work on contract management and the use of contractors to support deployed forces have resulted in numerous recommendations over the last several years. In addition, Congress has enacted legislation requiring DOD to take specific actions to improve its management and oversight of contracts. In response, DOD has issued guidance to address contracting weaknesses and promote the use of sound business arrangements. DOD has established a framework for reviewing major services acquisitions, promulgated regulations to better manage its use of contracting arrangements that can pose additional risks for the government, including time-and-materials contracts and undefinitized contracting actions, developed guidance on linking monetary incentives for contractors to acquisition outcomes, and has efforts under way to identify and improve the skills and capabilities of its workforce. These are positive steps, but inconsistent implementation has hindered past DOD efforts to address these high-risk areas. To improve outcomes on the whole, DOD must ensure that these policy changes and others are consistently put into practice and reflected in decisions made on individual acquisitions. We have ongoing work assessing DOD’s efforts to implement a service acquisition management approach, including its development of a structure for reviewing its major services acquisitions, as well as its use of different types of contract arrangements.

Section 801 of the National Defense Authorization Act for Fiscal Year 2002 required DOD to establish a management structure for the procurement of services, including developing a structure for reviewing individual service transactions, holding accountable employees responsible for procuring services, and collecting and analyzing service contract data. In addition, section 802 of the National Defense Authorization Act for Fiscal Year 2002 established a goal for DOD to use improved management practices to achieve savings in expenditures for procurement of services. In response to this requirement, DOD and the military departments established a service acquisition management structure, including processes at the headquarters level for reviewing individual, high-dollar acquisitions. The National Defense Authorization Act for Fiscal Year 2006 further developed the requirements for a management structure for the procurement of contract services. Among other things, the National Defense Authorization Act for Fiscal Year 2006 required DOD’s management structure to provide for the Under Secretary of Defense for Acquisition, Technology and Logistics (USDAT&L) to: establish contract services acquisition categories, based on dollar thresholds, for the purpose of establishing the level of review, decision authority, and applicable procedures identify the critical skills and competencies needed to carry out the procurement of services.

The National Defense Authorization Act for Fiscal Year 2006 also required the USDAT&L and senior acquisition management officials within the military departments to ensure that competitive procedures and performance-based contracting are used to the maximum extent practicable. In 2006, DOD updated its policies aimed at strengthening how it plans, manages, and oversees services acquisition in response to the legislation. Later, in December 2008, DOD incorporated its acquisition review thresholds for major services acquisitions in DOD Instruction 5000.02, Operation of the Defense Acquisition System.

The National Defense Authorization Act for 2008 required DOD to take additional actions to improve its visibility over the department’s reliance on services contractors as well as its management and oversight of its services acquisitions.

Section 807 required DOD to provide Congress an annual inventory of contractor-provided services, to include information on the missions and functions of the contractor, the number of full-time contractor employees paid for performing the activity, and the organization whose requirements are being met through contractor performance. In addition, this provision required the military departments to review the inventory to identify activities that should be considered for conversion to performance by DOD civilian employees or to an acquisition approach that would be more advantageous to DOD. The first inventory was to have been reported to Congress not later than June 30, 2008. At this time however, only the Army has begun the process to comply with this requirement. According to DOD officials, the Air Force and Navy will issue their prototype inventories in the third quarter of fiscal year 2009.

Section 808 required DOD to issue guidance and implementation instructions for performing periodic independent management reviews of contracts for services. In September 2008, DOD issued a policy memorandum to implement these reviews, referred to as peer reviews. Under DOD’s plan the Director, Defense Procurement, Acquisition Policy and Strategic Sourcing would be responsible for implementing reviews of acquisitions of services with an estimated maximum value of over $1 billion, while the DOD components would be responsible for reviews of acquisitions under $1 billion. In February 2009, DOD revised its guidance for how the review teams should conduct peer reviews to address pre-and- post-award review elements of the acquisition and the criteria that should be used to conduct these reviews. According to DOD officials, this guidance was developed as part of the agency’s response to some of the issues identified in our DOD contact management high risk area. We continue to monitor DOD’s implementation of these efforts.

In late 2008, DOD began an effort, directed by the Chairman of the Joint Chiefs of Staff, to examine the department’s use of service contracts in Iraq and Afghanistan. The purpose of this effort is to improve DOD’s understanding of the range and depth of contractor capabilities necessary to support the Joint Force. The study will address where DOD is most reliant on contractor support, informing longer term force structure issues such as the potential for increasing DOD’s military and civilian work force in order to in-source services currently provided by contractors.

We have also made numerous recommendations over the past 10 years aimed at improving DOD’s management and oversight of contractors supporting deployed forces, including the need for (1) DOD-wide guidance on how to manage contractors that support deployed forces, (2) improved training for military commanders and contract oversight personnel, and (3) a focal point within DOD dedicated to leading DOD’s efforts to improve the management and oversight of contractors supporting deployed forces.

In addition, Section 854 of the National Defense Authorization Act for 2007 directed the Secretary of Defense in consultation with the Chairman of the Joint Chiefs of Staff to develop joint policies for requirements definition, contingency program management, and contingency contracting during combat and post-conflict operations. The National Defense Authorization Act for Fiscal Year 2008 added a new requirement directing that these joint policies provide for training of military personnel outside the acquisition workforce who are expected to have acquisition responsibilities including oversight of contracts or contractors during combat operations, post-conflict operations and contingency operations.

As we reported in November 2008, while DOD has more to do in this area, it is developing, revising, and finalizing new joint policies and guidance on the department’s use of contractors to support deployed forces. Examples include: In October 2008, DOD finalized Joint Publication 4-10, Operational Contract Support, which establishes doctrine for planning, conducting, and assessing operational contract support integration and contractor management functions in support of joint operations. The joint publication provides standardized guidance and information related to integrating operational contract support and contractor management.

DOD is revising DOD Instruction 3020.41, Program Management for the Preparation and Execution of Acquisitions for Contingency Operations, which strengthens the department’s joint policies and guidance on program management, including the oversight of contractor personnel supporting a contingency operation.

DOD has also taken steps to improve the training of military commanders and contract oversight personnel. As we reported in November 2008, the Deputy Secretary of Defense issued a policy memorandum in August 2008 directing the appointment of trained contracting officer’s representatives prior to the award of contracts. U.S. Joint Forces Command is developing two training programs for non-acquisition personnel to provide information necessary to operate effectively on contingency contracting matters and work with contractors on the battlefield. In addition, the Army has a number of training programs available that provide information on contract management and oversight to operational field commanders and their staffs. The Army is also providing similar training to units as they prepare to deploy, and DOD, the Army, and the Marine Corps have begun to incorporate contractors and contract operations in mission rehearsal exercises.

In October 2006, the Deputy Under Secretary of Defense for Logistics and Materiel Readiness established the office of the Assistant Deputy Under Secretary of Defense (Program Support) to act as the focal point for DOD’s efforts to improve the management and oversight of contractors supporting deployed forces. This office has taken several steps to help formalize and coordinate efforts to address issues related to contractor support to deployed forces. For example, the office took a leading role in establishing a community of practice for operational contract support— comprising subject matter experts from the Office of the Secretary of Defense, the Joint Staff, and the services—that may be called upon to work on a specific task or project. Additionally, the office helped establish a Joint Policy Development General Officer Steering Committee to guide the development of the Office of the Secretary of Defense, Joint Staff, and service policy, doctrine, and procedures to adequately reflect situational and legislative changes as they occur within operational contract support.

In addition, DOD has efforts under way to identify and improve the skills and capabilities of its workforce. For example, in response to recommendations from the Gansler Commission, the Army proposed increasing its acquisition workforce by over 2,000 personnel. However, the Army also acknowledged that this process will take at least 3 to 5 years to complete. In addition, we continue to monitor DOD’s planned and completed corrective actions to address our audit report recommendations to improve its acquisition of services.

Concluding Observations

As the largest buyer of services in the federal government, and operating in an environment where the nation’s large and growing deficits require difficult resource decisions, DOD must maximize its return on investment and provide the warfighter with needed capabilities at the best value for the taxpayer. DOD has recognized that it faces challenges with contract management and the department has taken steps to address these challenges, including those outlined in this testimony. These challenges are daunting. While DOD’s recent initiatives may improve how the department plans service acquisitions at a strategic level, these efforts will not payoff unless DOD’s leadership can translate its vision into changes in frontline practices. At this point, DOD does not know how well its services acquisition processes are working and whether it is obtaining the services it needs while protecting DOD’s and the taxpayer’s interests. While DOD has generally agreed with our recommendations intended to improve contract management, much remains to be done. For example: In the near term, DOD must act forcefully to implement new procedures and processes in a sustained, consistent, and effective manner across the department. Doing so will require continued, sustained commitment by senior DOD leadership to translate policy into practice and to hold decision makers accountable.

At the same time, while the department and its components have taken or plan to take actions to further address contract management challenges, many of these actions, such as the Army’s efforts to increase its acquisition workforce, will not be fully implemented for several years. DOD will need to monitor such efforts to ensure that intended outcomes are achieved.

At the departmentwide level, DOD has yet to conduct the type of fundamental reexamination of its reliance on contractors that we called for in 2008. Without understanding the depth and breadth of contractor support, the department will be unable to determine if it has the appropriate mix of military personnel, DOD civilians, and contractors. As a result, DOD may not be totally aware of the risks it faces and will therefore be unable to mitigate those risks in the most cost-effective and efficient manner.

The implementation of existing and emerging policy, monitoring of the department's actions, and the comprehensive assessment of what should and should not be contracted for are not easy tasks, but they are essential if DOD is to place itself in a better position to deliver goods and services to the warfighters. Moreover, with an expected increase of forces in Afghanistan, the urgency for action is heightened to help the department avoid the same risks of fraud, waste, and abuse it has experienced using contractors in support of Operation Iraqi Freedom.

Mr. Chairman and members of the committee, this concludes our testimony. We would be happy to answer any questions you might have.

Contacts and Acknowledgments

For further information about this testimony, please contact John Hutton, Director, Acquisition and Sourcing Management, on (202) 512-4841 or huttonj@gao.gov or William Solis, Director, Defense Capabilities and Management, on (202) 512-8365 or solisw@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this testimony. Other key contributors to this testimony include Carole Coffey, Timothy DiNapoli, Justin Jaynes, John Krump, Christopher Mulkins, James A. Reynolds, Karen Thornton, Thomas Twambly, and Anthony Wysocki.

Appendix I: Selected GAO Products

High-Risk Series

High-Risk Series: An Update. GAO-09-271. Washington, D.C.: January 2009.

Contract Management

Defense Contracting: Army Case Study Delineates Concerns with Use of Contractors as Contract Specialists. GAO-08-360. Washington, D.C.: March 26, 2008.

Defense Contract Management: DOD’s Lack of Adherence to Key Contracting Principles on Iraq Oil Contract Put Government Interests at Risk. GAO-07-839. Washington, D.C.: July 31, 2007.

Defense Contracting: Improved Insight and Controls Needed over DOD’s Time-and-Materials Contracts. GAO-07-273. Washington, D.C.: June 29, 2007.

Defense Contracting: Use of Undefinitized Contract Actions Understated and Definitization Time Frames Often Not Met. GAO-07-559. Washington, D.C.: June 19, 2007.

Defense Acquisitions: Improved Management and Oversight Needed to Better Control DOD’s Acquisition of Services. GAO-07-832T. Washington, D.C.: May 10, 2007.

Defense Acquisitions: Tailored Approach Needed to Improve Service Acquisition Outcomes. GAO-07-20. Washington, D.C.: November 9, 2006.

Operational Contract Support

Contract Management: DOD Developed Draft Guidance for Operational Contract Support but Has Not Met All Legislative Requirements. GAO-09-114R. Washington, D.C.: November 20, 2008.

Contingency Contracting: DOD, State, and USAID Contracts and Contractor Personnel in Iraq and Afghanistan. GAO-09-19. Washington, D.C.: October 1, 2008.

Military Operations: DOD Needs to Address Contract Oversight and Quality Assurance Issues for Contracts Used to Support Contingency Operations. GAO-08-1087. Washington, D.C: September 26, 2008.

Defense Management: DOD Needs to Reexamine Its Extensive Reliance on Contractors and Continue to Improve Management and Oversight. GAO-08-572T. Washington, D.C.: March 11, 2008.

Defense Logistics: The Army Needs to Implement an Effective Management and Oversight Plan for the Equipment Maintenance Contract in Kuwait. GAO-08-316R. Washington, D.C.: January 22, 2008.

This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.