DLA is a DOD Combat Support Agency under the supervision, direction, authority, and control of the Under Secretary of Defense for Acquisition, Technology, and Logistics. DLA’s mission is to provide its customers—the military services and federal civilian agencies—with effective and efficient worldwide logistics support as required. DLA buys and manages a vast number and variety of items for its customers, including commodities such as energy, food, clothing, and medical supplies. DLA also buys and distributes hardware and electronics items used in the maintenance and repair of equipment and weapons systems. Customers determine their requirements for materiel and supplies and submit requisitions to any of four DLA supply centers. The centers then consolidate the requirements and procure the supplies for their customers. DLA provides its customers with requested supplies in two ways: some items are delivered directly from a commercial vendor while other items are stored and distributed through a complex of worldwide distribution depots that are owned and managed by both DLA and the military services. DLA refers to this ordering and delivery process as materiel management or supply-chain management. Figure 1 provides a snapshot of this process. Because DLA is the sole supplier for many critical items that can affect the readiness of the military services, the agency strives to provide its customers with the most efficient and effective logistics support. Thus, DLA has adopted a policy to provide customers with “the right item, at the right time, right place, and for the right price, every time.” In an effort to institutionalize this customer support concept, DLA has adopted the Balanced Scorecard approach to measure the performance of its logistics operations. The scorecard, a best business practice used by many private and public organizations, is intended to measure DLA’s performance by integrating financial measures with other key performance indicators around customers’ perspectives; internal business processes; and organization growth, learning, and innovation. Our work showed that customers at the eight locations we visited expressed satisfaction and dissatisfaction with the services the agency provides. On the one hand, customers are generally satisfied with DLA’s ability to quickly respond to and deliver requests for routine, high-demand, in-stock items; provide customers with an easy-to-use ordering system; and manage an efficient prime vendor program. On the other hand, customers at some locations were dissatisfied that, among other things, DLA is unable to obtain less frequently needed, but critical, items and parts and provide accurate and timely delivery status information. Some customers did not express an opinion on the overall quality of customer service. One aspect of DLA customer support is to provide customers with supplies when they need them. Common supplies include vehicle parts such as pumps, hoses, filters, and tubing. Timeliness, which sometimes requires deliveries to be made in a day or less, can vary with customers, depending on the particular item. However, customers at all locations we visited commented that they were generally satisfied with DLA’s ability to provide most supply items in a time frame that meets their needs. Customers stated that the majority of the routine, frequently demanded supplies they order through DLA are delivered quickly—a view that is also supported by a February 2002 DLA performance review. The review concluded that the majority of requisitions (over 85 percent) was filled from existing inventories within DLA’s inventory supply system. Similarly, a 2001 Joint Staff Combat Support Agency Review Team assessment of DLA’s support to the unified commands indicated that overall, DLA received outstanding comments regarding its ability to provide its customers with timely supplies and services. Customers were also satisfied with the ease in ordering supplies such as the pumps, hoses, and filters mentioned above. Customers stated that even though they conduct large amounts of business through DLA, they had few problems with the ordering process. This occurs because, according to some customers, ordering is facilitated by effective on-line systems that work well and have readily available information. Another method that DLA uses to ensure customer satisfaction is its prime vendor program, which DLA instituted to simplify the procurement and delivery of such items as subsistence and medical or pharmaceutical supplies that commonly have a short shelf life. The program enables customers to directly interact with vendors, thereby reducing the delivery time for these supplies. Two customers of these DLA-managed prime vendor programs told us the programs effectively reduced delivery time. For example, at one location, prime vendors reduced the delivery time of food items from 7 days—the time it took to deliver the items when purchased from DLA—to 2 days for items purchased directly from prime vendors. The customers we spoke with at a medical supply unit told us they were so pleased with the prime vendor’s quick delivery time that they intend to obtain even more medical supplies from the prime vendor. They also told us that the prime vendor provides an additional service in the form of monthly visits to assess customer satisfaction with its services. The unit pointed out that DLA’s customer support representatives are less likely to make such frequent visits. Although customers seemed pleased with the way DLA handles routinely available items, some raised concerns over the agency’s ability to provide critical items such as weapon system parts, timely and accurate information on the status of ordered items, and proactive management for high-priority requisitions. A Combat Support Agency Review Team assessment in 1998 also surfaced similar issues. Additionally, customers we talked to criticized how DLA manages customer-owned assets in DLA warehouses. As previously noted, DLA strives to provide the timely delivery of all supplies and parts, including common consumable supply items like food; clothing and hardware; and critical parts for weapons systems such as tanks, helicopters, and missiles. Customers at four locations we visited told us that DLA was not able to timely deliver some critical items, such as weapons systems parts, which significantly affected their equipment readiness. A number of customers told us that the items they have difficulty obtaining from DLA are those that are more costly or infrequently required. At two locations, customers used parts from existing equipment (known as “parts cannibalization”) because they were unable to obtain the parts they needed. At two other locations, customers said they grounded aircraft and/or deployed units without sufficient supplies. Customers at one location experienced an over-6-month delay in obtaining helicopter parts. As a result, customers at this location told us that some of the unit’s helicopters were unable to fly their missions. We reported in November 2001 that equipment cannibalizations adversely affect the military services, resulting in increased maintenance costs, and lowered morale and retention rates because of the increased workload placed on mechanics. One customer also told us that DLA does not provide adequate information about items requiring long procurement lead times. The customer stated that having this information more readily available would aid customers in making decisions about the types and quantities of items they should retain to minimize the impacts of long DLA lead times. The 1998 Combat Support Agency Review Team’s assessment conducted at military service field activities found that even though DLA met its overall supply availability goal of 85 percent, the remaining 15 percent of items that were not available “almost certainly includes a number of items that are critical to the operation of essential weapon systems.” The assessment attributed this shortfall to flaws in DLA’s requirements determination models, which are used to estimate customers’ demands so that DLA can maintain sufficient inventory quantities. The study further stated that customers are not satisfied with the delivery time for items that are not in stock. In fact, in April 2002, the overall logistics response time was almost 100 days for nonstocked items—a problem that appears to have persisted for the last several years, in spite of efforts to reduce this time. Customers at four locations provided us with examples of back-ordered items having lead times in excess of 1 year, such as navigational instruments and airframe parts. In discussing this issue further with DLA headquarters officials, they acknowledged that this is a problem and are working on a number of initiatives to address customers’ concerns. Customers need accurate and timely information on the status of their orders so they can plan equipment maintenance schedules to optimize the readiness of existing equipment. However, customers at six locations were frustrated with obtaining accurate and timely information from DLA item managers and the automated systems that are intended to provide status information on requisitions. Customers at three locations said that when they tried to directly contact item managers by telephone, the managers often could not be reached and voice-mail messages were seldom returned. Furthermore, military service customers told us that DLA’s automated requisition systems often do not contain accurate status data. Of particular concern to customers are the expected shipping or delivery dates posted on the automated systems. These dates show when parts will be available and allow units to coordinate maintenance schedules. If the dates are incorrect, units cannot effectively plan to have equipment available to be repaired. We discussed this concern with DLA headquarters officials, who told us they are investigating the problem. Another significant concern raised by customers at three locations was that DLA is not proactive in seeking alternate ways to obtain critical items that are not immediately available within DLA’s supply system. DLA typically places such items on back order, which, to meet mission needs, places a burden on customers to find their own means to obtain the necessary items right away. A number of customers at these three locations said they felt that DLA, in an effort to be more customer focused, should do more to seek out alternate sources of supply to alleviate these high-priority back orders. Some customers also remarked that the required efforts for them to call vendors and solicit bids is a problem for their unit because of limited staffing levels and lack of contracting capabilities. In one instance, an aviation supply unit requisitioned a critical part from DLA that was needed to repair a helicopter unable to fly its mission. This requisition was placed on back order by DLA, and delivery was not expected to occur until 8 months later. Because of the critical nature of the needed part, the unit had to search for other means to obtain the part sooner. In fact, the unit directly contacted the same vendor that DLA was working with to fill the back orders and learned that the vendor had stock on hand and would be able to ship the item immediately. The unit subsequently purchased the part from that vendor instead of waiting for it to be available from DLA. In another instance, a DLA item manager informed an aircraft maintenance depot customer that $2 million worth of critical parts for a helicopter engine overhaul program would be placed on back order because the parts were not available from the DLA vendor. In researching listings for property to be disposed of, the customer found the required parts—still new and unopened in the manufacturers’ container—available for redistribution or sale within DLA’s disposal system. As a result, the customer initiated a shipping request to procure the $2 million in helicopter parts for only the cost to ship the items. DLA manages all warehousing functions at locations where a DLA distribution depot is collocated with a military activity. Management functions include, among other things, logging in and storing equipment. During the course of our interviews, customers raised concerns over DLA’s handling of these functions. At three of the sites we visited, the customers perceived that their assets were not being serviced and maintained as required. Their concerns centered on DLA’s process for recording the ownership of equipment and the commingling of different customers’ inventories. To assign asset ownership, DLA “codes” items in its automated inventory system. That is, DLA assigns unique codes to differentiate between Army, Navy, Marine Corps, Air Force, and DLA-owned assets. However, customers at three locations we visited stated that in numerous instances, DLA assigned inventory items to the wrong management account, thus creating the possibility that an item ordered and paid for by one unit or service could be issued to another. One location we visited had documented over $1 million worth of items coded into the wrong management account. Another location identified $621,000 worth of incorrectly coded items. Before the errors were corrected, neither activity could access the materials they needed. As a result, both locations invested unnecessary amounts of time and money in correcting DLA’s error. During our review, we brought this issue to the attention of DLA officials, who indicated that they would investigate the problem. Customers also expressed concerns about the commingling of service- owned assets with DLA-owned assets in DLA-managed warehouses. Like inaccurate coding, commingling creates a significant risk that items will be issued by the warehouse to someone other than the purchasing unit. As a result, the items would not be available to the true owner when needed. Also, for equipment items that need periodic inspection and repair, there is a risk the owner will expend resources to perform maintenance or repairs but not be able to retrieve the item because DLA mistakenly issued that item to a different requisitioning entity or military service. As a result, the “true owner” could have needlessly spent resources on items given to somebody else and also be left with items still needing repair. In discussions with DLA headquarters officials, they acknowledged the problem and told us that DLA is taking steps to address it with a National Inventory Management Strategy, which is part of DLA’s goal to better manage its supply chain effectiveness. DLA’s approach for obtaining customer service feedback has been of limited usefulness because it lacks a systematic integrated approach for obtaining adequate information on customer service problems. As a result, the agency does not have the information necessary to identify its customers’ concerns, and more importantly, to initiate actions for improving customer service, thereby placing at risk DLA’s ability to meet its overall goal of providing quality service to the war fighter. In particular, DLA has not (1) adequately identified all of its customers, (2) effectively solicited customer feedback, and (3) clearly identified those accountable for ensuring customer satisfaction. Obtaining good meaningful feedback from customers means knowing who those customers are. DLA broadly defines a “customer” as someone who purchases items or directly causes products to be bought, but DLA has not identified who those individuals are from the multitude of organizations it deals with. DLA’s current portfolio of customers is identified by approximately 49,000 address codes, known as DOD Activity Address Codes (DODAACs). The military services assign DODAACs to various organizations and activities for ordering supplies. However, these address codes, a legacy of a system built in the 1960s, contain little information about the customer’s organization beyond a physical address. No meaningful customer contact point is associated with the codes or, in many cases, a specific organization that DLA can use as a basis for interaction with the customers using their services. As a result, DLA has no effective process to initiate and maintain contact with its customers for soliciting feedback. Without such a customer interface process, DLA has no routine means to understand customers’ needs and to take appropriate corrective actions to address those needs. Our efforts to identify and interview DLA customers were hindered because a single DODAAC does not necessarily equate to a single customer. In many cases we found that one organization interacts with DLA using a number of DODAACs. For example, DLA’s customer database shows over 580 DODAACs for Fort Bragg. However, according to DLA and Army officials, the number of Fort Bragg customer organizations interacting with DLA for these same DODAACs is smaller. The reason for this is that, in part, central order points at Fort Bragg are responsible for submitting and tracking orders for a number of smaller organizations, thereby covering multiple DODAACs. In addition, each of these organizations also uses multiple DODAACs to differentiate between various types of supply items, such as repair parts and construction materials. For example, one DODAAC is used for ordering numerous repair parts while another is used for ordering construction materials. One of these customer organizations at Fort Bragg is the Division Support Command of the 82nd Airborne Division, which interacts with DLA for supplies ordered using 159 different DODAACs. Thus, many DODAACs could represent only one customer. Figure 2 illustrates the relationship between the DODAACs used by DLA to define customers and the Division Support Command. A principal aspect of DLA’s strategic plan is for managers to focus on customers’ needs and improve customer satisfaction by listening to customers about the quality of service they receive—both good and bad— and making changes necessary to enhance that service. DLA uses customer surveys, customer support representatives, and focus groups to obtain feedback from its customers on their level of satisfaction with the services DLA provides. For example, DLA conducts quarterly mail-out surveys to measure overall customer satisfaction levels. It also places customer support representatives at selected customer organizations to assist customers in planning, implementing new supply initiatives, and solving problems. However, we noted several weaknesses in these methods. Specifically, (1) the satisfaction survey response rates are too low to provide meaningful statistical analyses of customer satisfaction, (2) the survey instrument does not provide a sufficient means to understand why customers may be less than satisfied, and (3) customer support representatives are more reactive than proactive in soliciting customer feedback. The quarterly mail-out surveys that DLA uses to measure customer satisfaction elicit a relatively low number of responses from DLA customers, significantly limiting its usefulness in soliciting customer feedback. The survey response rates were too low to provide meaningful statistical analyses of customer satisfaction. The response rate for the 33,000 surveys that DLA mailed out in fiscal year 2001 averaged around 23 percent, and only about 20 percent for the August 2001 cycle (the latest cycle where results have been made available). As such, less than one quarter of DLA’s customers are providing input on how they perceive DLA support and what problems they are experiencing that may need to be addressed. Large survey organizations like Gallup attempt to get response rates of between 60 and 70 percent for their mail surveys. Experts on customer satisfaction measurement have stated that although survey response rates are never 100 percent, an organization should strive to get its rate as close as possible to that number. They suggest that ideally, organizations can obtain response rates of over 70 percent. The experts also noted that organizations conducting surveys commonly make the mistake of assuming that if a final sample size is large, the response rate is unimportant. This leads organizations to accept response rates well under 25 percent. However, such low rates can lead to serious biases in the data. Having an inadequate understanding of who its customers are likely contributes to DLA’s problem with low response rates. The surveys are mailed to addresses associated with the DODAACs and include with each survey a message asking that the survey be provided to a person most familiar with requisitioning and ordering supplies. However, during the fiscal year 2001 survey period, over 2,200 of the 33,000 surveys mailed (about 7 percent) were returned to DLA as “undeliverable” or were delivered to people who were no longer customers. Furthermore, another 128 respondents noted in their survey returns that they do not consider themselves to be customers. DLA officials stated that the undeliverable rate increases when there are many units that move to other locations or when service officials do not update DODAACs for changed addresses. The quarterly mail-out survey asks customers to rate their overall satisfaction with DLA products and services, along with specific aspects of support, such as providing products in time to meet needs and effectively keeping customers informed. While these surveys provide general aggregate information on the levels of customer satisfaction, they do not provide the means to understand why customers may be less than satisfied. For example, a number of customers we interviewed voiced concern over the fact that status dates for back-ordered items were either sometimes wrong or varied between different inventory systems. The survey might indicate only an overall low level of satisfaction in the area of keeping customers informed but would not provide a reason. If this problem were systemic throughout DLA, there would be less of an opportunity to take immediate corrective action. Most recently, in June 1999, DLA supplemented a quarterly survey with two focus groups targeted at soliciting specific customer feedback on DLA’s communication efforts. While DLA determined the focus groups to be an excellent feedback mechanism, the sample size was too small for DLA to run a statistical analysis of the data obtained; and the topics for discussion were limited to customer communication. DLA officials stated that they use a number of methods to obtain customer feedback. These include analyses of survey results, focus groups, and structured interviews. However, they acknowledged that the usefulness of these methods is somewhat limited owing either to low response rates; limited discussion topics; small sample sizes; or, in the case of structured interviews, the fact that the most recent ones were conducted in 1997. DLA’s own survey results also indicate the flaws with its survey techniques. For example, DLA’s fiscal year 2000 survey results show that customers rated as “low satisfaction” their ability to reach the right DLA person to meet their needs. However, the survey noted that “due to its high importance to customers and the myriad of interpretations of ‘less than satisfied’ responses to this attribute, more information will need to be gathered” to determine what issues are preventing customers from reaching the right person. This indicates that DLA’s survey was not adequate to get behind the underlying causes of customer dissatisfaction. In fact, with respect to low satisfaction ratings, the survey reports for fiscal years 2000 and 2001 recommended that DLA conduct one-on-one interviews to identify why customers were not satisfied with DLA services. Another difficulty that DLA encounters in using mail-out satisfaction surveys to identify customer problems is that the surveys are designed to protect the confidentiality of the respondents, which limits DLA’s ability to follow up with customers for adequate feedback. As a result, there is no means to follow-up with customers expressing low satisfaction levels to identify specific problems or to determine what, if any, corrective actions are needed. During our meetings with DLA customers, we were able to identify specific problems only by engaging in a dialogue with them about their experiences. In conducting these in-depth discussions on aspects of the supply process such as placing orders, obtaining the status of outstanding requisitions, receiving supply items, and obtaining customer service, we were able to ask follow-up questions to determine exactly what problems they were experiencing in some of these areas. Another method DLA uses to facilitate customer service is the placement of customer support representatives at key customer locations. The use of these on-site representatives has the potential to provide DLA with a good link to its customers. In fact, some customers at three locations we visited specifically noted their satisfaction with the assistance the representatives provided. However, according to DLA headquarters officials, customer support representatives have been more reactive in that they help customers resolve only specific problems or assist in implementing new initiatives as requested. DLA headquarters officials told us that the representatives neither proactively solicit feedback on a regular basis from the multitude of customers in their geographical area nor reach out to identify the types of problems customers are experiencing. Furthermore, not all representatives are in contact with all DLA customers at their assigned locations. For example, at one location we visited, the representative was working closely with a specific customer organization. According to officials at this location, the representative has been very helpful to them in resolving supply problems and implementing new initiatives. However, a number of other customers at this location said they do not use the customer support representative at all because they use other options, such as call centers. Some customers noted that they were not even aware that there was such a representative in the area. The Combat Support Agency Review Team’s assessment in 1998 also found that some customers were unaware that customer support representatives even existed. The study identified a need for DLA to improve its interaction with customers and suggested that DLA “get out more and visit the customers” to identify and correct problems. Headquarters officials told us they assign customer support representatives to DLA’s larger customers, which account for about 5 percent of the overall customer population and 80 percent of the agency’s business. Officials also stated they recognize that the customer support representative program is not as effective as it should be. As a result, the agency currently has initiatives under way to (1) provide more customer support representatives and training, (2) standardize the representatives’ roles, and (3) make the representatives more proactive in serving customers. An important part of providing effective customer service is simplifying customers’ access to the organization, such as through centralized contact points. In addition, best practices research emphasizes the need for a single, centralized management framework for receiving customer feedback so that all information about the customers can be linked together to facilitate a more complete knowledge of the customer. However, DLA does not provide a “single face” to its customers for addressing their issues. To obtain assistance, customers sometimes need to navigate through a number of different channels, none of which are interconnected. This process causes confusion with customers and fragmented accountability throughout DLA for customer satisfaction. When customers order multiple types of supply items, they must use many channels, depending on the type of item, to obtain assistance from DLA. However, as DLA has noted, there is no single DLA contact point responsible for resolving customers’ problems for all the items they requisition. For example, the supply centers are responsible for managing specific weapons system parts or types of commodities. As such, problem resolution is performed through each supply center, depending on the type of item the customer is ordering. To obtain assistance with requisitions, customers must contact the appropriate supply center, generally through its customer “call center,” which is an activity dedicated to provide customer assistance for the particular items. In addition, Emergency Supply Operation Centers are available at each supply center for high- priority items. Also, customers can contact individual item managers at the supply centers to resolve problems with their orders. At three locations, some customers told us they are sometimes confused over whom to call and reported difficulties with getting in touch with the right person to resolve their problems. Customers at four locations were also frustrated with the quality of assistance provided by DLA, noting that while some of the DLA representatives were helpful, others were not able to give them the assistance they needed. To illustrate further, one aviation supply unit we visited had high-priority, back-ordered requisitions from each of the three DLA supply centers in Richmond, Virginia; Columbus, Ohio; and Philadelphia, Pennsylvania. As a result of these back orders, some of the unit’s aircraft were unable to operate because of maintenance needs. In order to get assistance with these requisitions, either to request help in expediting the order or to obtain better status information, unit supply personnel needed to contact the call centers or the Emergency Supply Operation Centers at each of the supply centers, depending on the item. If there were a single DLA point of contact, the unit could go to that contact for assistance with all the items on its list of priority requisitions. Another problem with DLA’s having many separate lines of communication with its customers is that meaningful information about those customers is not collected centrally for analysis. For example, each of the supply centers accumulates vital information about customer satisfaction through its contacts with customers. For instance, customers express specific problems they are having when getting help through the call centers. They might also convey information on problems they are having to various supply center teams conducting on-site visits for purposes of training or other liaison activities. However, this information is neither shared between the supply centers nor provided to the DLA corporate level for a global review. As a result, no analysis of this information can be made to identify systemic problems or any accountability at one place for a given customer to ensure that its concerns are being addressed. While DLA has initiatives under way to improve its customer service, there are opportunities to enhance these initiatives to provide for an improved customer feedback program. DLA has recognized that it is not as customer focused as it should be and is developing a new strategy to improve its relationship with its customers. This new strategy, referred to as the Customer Relationship Management initiative, lays out an improved approach to customer service that creates a single DLA face to customers and focuses on customer segments to develop a better understanding of the customer. However, DLA’s initiatives do not completely address the limitations we identified in its current approaches for obtaining customer service feedback, such as by improving the way that it solicits feedback from individual customers. Research on best practices for customer service shows that successful organizations utilize multiple approaches to listen to their customers. These approaches include transaction surveys, customer interviews, and complaint programs that provide qualitative and quantitative data. The research also points to a need for centrally integrating all customer feedback so that managers can achieve a better understanding of customers’ perceptions and needs. In February 2002, DLA’s Deputy Director stated that DLA “has been internally focused rather than customer focused” and that its culture has been to talk to customers only “when problems arose.” To address this problem, DLA has begun a multimillion-dollar initiative aimed at focusing its business operations to better deliver important customer outcomes and actively managing relationships with its customers. This effort, known as Customer Relationship Management, is being developed in conjunction with DLA’s broader strategic planning initiatives such as Business Systems Modernization and implementation of the Balanced Scorecard approach to performance measurement. To implement Customer Relationship Management, DLA expects to spend about $73 million during fiscal years 2002-2008. According to DLA officials, when this effort is complete, DLA expects its customer service program to be on the same level as those in place at leading organizations in the private sector. The concept of the Customer Relationship Management initiative is a step in the right direction toward significantly improving DLA’s relationship with its customers. For example, part of the management initiative is a plan to radically change the focus of its business practices and improve its interactions with customers. To do this, DLA is grouping customers by business segment, collaborating with these segments to achieve a better understanding of their needs, and tailoring logistics programs to the unique needs of the segments. Examples of business segments include deployable combat forces, industrial facilities, and training activities. Table 1 illustrates the proposed customer segments, which will include major military service commands. In an effort to streamline the numerous customer-reporting channels currently in place, DLA plans to establish a multilevel-focused account manager structure and increase accountability. DLA hopes that this effort will reduce the number of channels a customer must navigate to obtain assistance and focus accountability for customer satisfaction on account managers rather than on item managers. DLA plans to establish account managers at three levels: National Account Managers are to collaborate with military services at the departmental level, for demand planning and problem resolution. Customer Account Managers are to be the “single DLA face” to each customer segment. These managers are to collaborate with executives at the segment level to develop service-level agreements that outline customer segment needs and to resolve issues at the segment level. Customer Support Representatives are working-level DLA personnel who, on a day-to-day basis, work with specific customers within a segment, providing on-site assistance as appropriate. In addition, DLA plans to place its existing customer contact points, such as call centers and Emergency Supply Operation Centers, under the control of account managers instead of the supply centers. Although the Customer Relationship Management initiative is conceptually sound, the program’s implementation actions do not completely address the limitations we identified in its current practices. For example, the new strategy does not lay out milestones for implementing the program or specific improvements on how DLA solicits detailed feedback from its individual customers on their perceptions of service and the specific problems they are experiencing. The strategy also does not include a process for developing actions in response to issues that customers have identified and involving customers in that process. Furthermore, even though the plans include making account managers responsible for collecting customer feedback and exploring the idea of using Web-based tools to obtain customer feedback, they do not lay out specific tools or processes to accomplish this. To further illustrate, under the new Customer Relationship Management plan, an account manager would be created with responsibility for all customers within the U.S. Army Forces Command, which represents the Army’s deployable forces segment. (See table 1.) This manager would work with the Army’s customer representatives to identify customers’ needs at the Forces Command level and reach formal agreements on service. However, there is no revised set of tools in the plan for collecting detailed feedback on an ongoing basis from the individual customer organizations representing the more than 6,600 DODAACs (address codes that represent mailboxes, locations, or people) in the Forces Command. Furthermore, the improvement initiatives do not provide for actions to link military service customer DODAACs to specific accountable organizations. Under the Customer Relationship Management program, DLA has developed a customer profile database that links DODAACs to major military commands, such as the U.S. Army Forces Command. It also plans to link each DODAAC to a business segment through this database sometime in the future. However, as noted previously, the major command and business segment levels comprise numerous DODAACs. Interaction with customers to get detailed feedback on their level of satisfaction requires better identification of customer organizations beyond the data currently associated with a DODAAC. Studies examining best practices in the area of customer service have found that leading organizations use multiple approaches to listen to their customers’ concerns. In particular, a 2001 Mid-American Journal of Business study pointed out that best practice companies use multiple tools to gather these data rather than relying on a single method such as a customer survey, which might be too narrow in scope and limited in its application to fully capture customers’ concerns. The 2001 Mid-American Journal study and others concluded that the best approach for obtaining customer feedback is to use a broad measurement system with more than one listening tool to capture customers’ input from many different perspectives. Using different tools alone is not enough to effectively obtain customer feedback. Centrally linking the feedback obtained is also important. Best practices research shows that information obtained through various methods needs to be integrated in order to gain a more complete understanding of customers. Thus, by linking all the various feedback tools in a standard and consistent manner, the organization would have better diagnostic information to guide improvement efforts. On the basis of our discussions with private sector experts and our reviews of literature on customer service best practices, leading organizations such as AT&T WorldNet Services, U.S. West, and Eastman Chemical combine quantitative and qualitative listening tools to obtain customer feedback and then centrally integrate the data in one location. Quantitative tools include such methods as customer satisfaction surveys and customer complaints, which can provide measurable data for use in performance scorecards. Qualitative tools include focus groups, personal interviews, and observation and are used by organizations to provide a more in-depth understanding of their customers. According to the research, not all tools are appropriate for all organizations, and the research points out that careful selection is therefore important. Examples of “listening” tools being used by the best practices organizations we identified through our reviews of best practice studies follow: Customer satisfaction surveys. Research shows that most major organizations use listening tools such as relational and critical incident surveys to periodically capture customers’ overall perceptions about their organization and to measure satisfaction with specific transactions soon after they occur. These surveys can be administered through the mail, such as with DLA’s quarterly satisfaction survey; by telephone; in person; or electronically via the Internet. However, feedback from mail and electronic-based surveys can be more limited than that obtained through other methods because there is no opportunity to probe the respondent for better, more-detailed information. AT&T WorldNet Services, U.S. West, Eastman Chemical, and Hewlett-Packard are among the leading organizations that are turning to critical incident surveys in conjunction with other tools to learn more about customers’ perceptions. Critical incident surveys are becoming more popular in the private sector because they provide information related to specific processes, which can be used to make specific improvements. Customer complaints. Gathering complaint data is a standard practice for most companies. All aspects of the customer complaint process are measured and tracked through this mechanism. Information collected and analyzed from this approach includes the nature of the complaint, speed of resolution, and customer satisfaction with the resolution. Eastman Chemical, for example, uses customer complaint data in conjunction with a survey tool to obtain customer feedback. It organizes the complaint data along the same attributes as the survey data. Benchmark surveys. Benchmark surveys gather perceptions of performance from the entire market. These surveys usually gather customer perceptions of performance about top competitors in an industry. This allows the company to examine its customer-perceived strengths and weaknesses in the overall marketplace. Best practices companies, such as Sun Microsystems, use this information primarily in the strategic planning process to identify their competitive advantage in the marketplace and to identify opportunities and shortfalls in the industry. While continuous improvement may be a result of this listening tool, the real value, according to the research in this area, comes from breakthrough thinking to gain a sustainable advantage. Won-lost and why surveys. “Lost” customers—those who do not replace orders with a company—can be an excellent source of valuable information. Some companies, such as Eastman Chemical, employ “won- lost and why” surveys to measure actual customer behavior and the rationale behind the behavior. This survey is utilized on a current basis, being administered to customers soon after they are “won” or “lost” (i.e., decide to drop a company). For example, if a customer is won or lost, the company then probes the customer as to why its business was won or lost. For companies with a large number of customers, this tool may be implemented in a survey. Focus groups. Organizations use focus groups to get better information from customers than survey results provide. In these groups, customers are probed about why they answered survey questions the way they did. DLA has used focus groups to get detailed feedback on a single topic, but as noted previously, the number of individuals making up the focus groups was too small to draw agency-wide conclusions. AT&T Universal Card Services (now part of Citigroup) conducts multiple focus groups per year to discuss a wide range of topics. In these forums, both satisfied and dissatisfied customers discuss the company’s service, products, and processes. Customer interviews. Conducting interviews with customers can provide a way to get very detailed information about their specific needs and problems. Like focus groups, this tool is used by leading customer service organizations to probe survey respondents as to why they answered survey questions a certain way. U.S. West identifies dissatisfied customers from its surveys and follows up with them to determine what problems they are having and how they can be fixed. Customer observation. In performing observations, organizations send teams to visit customers where they observe how those customers interact on a daily basis with the organization. This tool complements verbal information obtained through customer interviews and focus groups in that it provides confirmation to and a deeper understanding of that information. Management listening. Using this tool, managers listen in on actual customer calls to the organization to learn first-hand about what customers are experiencing. In an example of this technique, one best practice company encourages all of its managers, including the chief executive officer, to listen to customer calls. Customer service representatives. Collecting information from those employees who are in continuous direct contact with customers provides valuable information to best practice organizations. Often, these representatives are among the first to recognize customer problems. As mentioned previously, DLA uses customer support representatives to obtain feedback. However, according to DLA officials, it does not currently have enough representatives assigned to its customers, and the representatives generally are not proactive in obtaining customer feedback. Furthermore, while DLA’s representatives provide headquarters with monthly written reports on customer support, best practice organizations have taken this a step further by using electronic feedback mechanisms. Research shows that best practice organizations have their customer service representatives gather ideas, perceptions, and opinions from customers and report them electronically through a corporate intranet system. These data are then coded and distributed throughout the organization, thereby centrally integrating the feedback information. Figure 3 shows an example of how multiple approaches can be linked, as illustrated by AT&T Universal Card Services’ use of a “Customer Listening Post” team. While high-quality service to its customers is an overall goal, DLA lacks the information necessary to systematically assess the quality of service it is providing its customers with. Indications are that customers, while satisfied in some areas, are dissatisfied in others. The failure to address areas of dissatisfaction means opportunities to improve supply readiness are being missed. DLA is in the process of developing a program to improve its customer service relationships, but it currently does not have in place an effective mechanism that systematically gathers and integrates information on customer service views so that solutions can be identified to better meet their needs. The agency’s current practices do not always surface these concerns, or more importantly, provide information on why they exist or how they can be corrected. To its credit, DLA is undertaking a number of initiatives to improve the effectiveness of its customer relationship improvement efforts. However, these initiatives do not completely address the limitations of its current approaches for obtaining customer feedback because DLA (1) has not yet fully determined who its customers are or how best to serve their needs; (2) has not established the means to determine the underlying causes for customer dissatisfaction in order to fully achieve its strategic goal of providing customers with the most efficient and effective worldwide logistics support; and (3) lacks a centralized, customer-driven integrated framework in which to solicit feedback from its customers. Also, customer mail-out surveys are insufficient for identifying the causes of customer dissatisfaction. Finally, DLA is not yet making full use of best practice techniques, as discussed in this report, to identify and address customers’ concerns. To improve DLA’s ability to determine its customers’ needs, identify solutions for better meeting those needs, improve the supply readiness of military units, and improve the efficiency and effectiveness of depot maintenance repair activities, we recommend that the Secretary of Defense direct the Under Secretary of Defense for Acquisition, Technology, and Logistics to require the Director of DLA, as part of the agency’s customer relationship improvement efforts, to take the following actions: Develop a comprehensive plan for obtaining customer feedback that includes but is not limited to the following actions: Work with the military services to arrive at a mutually agreed determination of the military organizations that function as DLA “customers.” In doing so, both DLA and the services should identify officials accountable for providing and receiving customer feedback. Develop a customer feedback program that uses a variety of approaches such as those depicted in the best practices research discussed in this report. In developing this program, pilot tests could be used to determine which approaches meet agency and customer needs. Establish milestones for implementing the customer feedback program and for identifying the office accountable for its implementation. Integrate all customer feedback into an overall assessment to provide managers with a better understanding of customers’ perceptions and concerns. Establish a process for developing actions in response to issues that are identified from the customer feedback program and involve customers in that process. Establish processes for providing customers with information on actions that are being taken to address customer feedback issues. Improve the usefulness of its customer survey instruments by identifying ways to improve customer response rates, such as the use of effective follow-up procedures. Clarify guidance for customer support representatives to ensure that they are responsible for routinely contacting customers to obtain customer feedback. We also recommend that the Secretary of Defense direct the Secretaries of the Army, Navy, and Air Force to identify specific organizations that will be responsible for working with DLA in establishing a mutually agreed determination of those activities, organizations, and individuals that function as DLA “customers” and for working with DLA as it implements its customer feedback program. The Department of Defense provided written comments on a draft of this report, which are reprinted in their entirety in appendix II. DOD generally concurred with our recommendations and agreed that DLA needs to increase its focus on customer satisfaction. The department also noted that DLA is taking or is planning to take a number of actions to respond to our recommendations. For example, under DLA’s Customer Relationship Management program, DLA National Account Managers are to identify customer organizations in concert with their military service negotiating partners. In addition, DOD intends to use its Defense Logistics Executive Board as a forum to obtain input from each of the services on the specific organizations that will be responsible for working with DLA on customer feedback issues. Furthermore, DLA intends to better integrate customer feedback into an overall assessment and to improve its processes for providing customers with information on actions that are being taken to address customers’ issues. DOD did not agree with our recommended action that DLA develop a customer feedback program that uses a variety of approaches, such as those depicted in the best practices research discussed in this report. DOD stated that DLA’s use of feedback mechanisms should not be dictated by the best practices research we discussed. It further stated that DLA should continue to have the latitude to use its customer satisfaction measurement resources in the most efficient manner. Our discussion of best practice approaches was only intended to illustrate various techniques that some best practices organizations use to improve the ways they collect and analyze customer feedback. It was not our intent to prescribe specific approaches that DLA should use. Rather, we included examples of some of the approaches to best illustrate the concept of using multiple and integrated customer feedback approaches to better listen to customers’ opinions and concerns. We continue to believe that DLA’s customer feedback program could benefit from studying best practice organizations, such as those discussed in this report as well as others, to identify additional feedback approaches that could be pilot-tested and implemented to help strengthen its current customer feedback efforts. We are sending copies of this report to the Secretary of Defense; the Secretary of the Army; the Secretary of the Navy; the Secretary of the Air Force; the Commandant of the Marine Corps; the Director, Defense Logistics Agency; the Director, Office of Management and Budget; and other interested congressional committees and parties. We will also make copies available to others upon request. In addition, the report will be available at no charge on the GAO Web site at http:// www.gao.gov Please contact me on (202) 512-4412 if you or your staff have any questions concerning this report. Major contributors to this report are included in appendix III. To determine how customers perceived the quality of service they received, we examined customer satisfaction studies and surveys such as the Defense Logistics Agency’s (DLA) fiscal year 2000 and fiscal year 2001 quarterly satisfaction surveys and the Joint Staff Combat Support Agency Review Team’s 1998 and 2001 assessments. In addition, we performed a case study analysis using a judgmentally selected sample of DLA customers that included the use of structured interviews to identify customers’ perceptions and levels of satisfaction with DLA service. The details of our customer selection process, interview techniques, and sampling methodology follow: We initially selected customers using DLA-provided databases of its “top” military customers, which DLA primarily based on sales volume. DLA identified customers by Department of Defense Activity Address Codes (DODAACs) or military installation. We compiled the DLA information into a single database that included over 800 customer records accounting for about $5.6 billion of DLA’s total $7.8 billion nonfuel supply sales (about 72 percent) to the military services for fiscal year 1999, the most recent available data at the time of our review. We judgmentally selected customers from the database to maximize our coverage of the following significant variables: dollar sales, geographic location, DLA-defined customer type (i.e., deployed and deployable forces, industrial organizations, training activities, and the “other” segment), commodity type, and military service branch. We did not validate the accuracy of the DLA sales data, since the data’s purpose was to provide us with general customer sales activity. Because the DLA-provided customer DODAAC and installation data did not provide us with sufficient information about specific customer organizations and related points of contact, we held discussions with DLA and military service officials to further define customers and subsequently visited those customer organizations and activities. We conducted over 50 structured interviews with customers at more than 20 selected activities. We designed the interview questions on the basis of aspects of DLA’s supply process: submitting requisitions, following up on the status of open requisitions, contacting DLA for customer service, and receiving supplies. We also discussed other factors related to DLA support, such as the availability, price, and quality of DLA-provided supply items. Some customers did not express an opinion on the overall quality of customer service. Our initial sample of DLA customers included customers from more than 20 locations throughout the continental United States and overseas, covering multiple customer types within each military service. However, because of the September 11, 2001, terrorist attacks on the World Trade Center in New York, and the Pentagon in Washington, D.C., we did not complete our planned visits. As a result, we limited our visits to eight military service customer locations within the continental United States, as shown in figure 4. Our selection of customers included all four military services and each of the DLA customer types except for deployed forces. Because we did not draw a statistical sample and we limited our selection of customers, the results of our work cannot be projected to DLA as a whole. However, DLA surveys, Combat Support Agency Review Team assessments, and comments from DLA officials suggest that many of the issues we raise are systemic problems. To determine how useful the agency’s approaches are for obtaining customer service feedback, we met with DLA headquarters officials to discuss current processes and planned initiatives for measuring customer service and obtaining feedback. We also discussed with DLA customers, feedback mechanisms such as the use of DLA customer support representatives and quarterly surveys. We reviewed relevant reports, briefing documents, and other key information related to the agency’s processes and mechanisms for soliciting customer feedback. Additionally, we examined the agency’s customer feedback survey techniques and methods, such as the use of quarterly mail-out surveys and focus groups. Furthermore, we conducted an extensive literature search of best practice organizations to determine popular techniques for collecting customer feedback, and their advantages and disadvantages. To determine whether there are opportunities to enhance DLA’s initiatives to improve customer service, we performed a comparative analysis between DLA’s current practices and planned initiatives, and best practices that we identified through extensive literature searches. We reviewed related DLA planning documents and met with agency officials to discuss the agency’s plans. Through our literature search, we identified relevant research performed in the area of best practices in customer satisfaction. We reviewed a number of pertinent studies and held discussions with customer satisfaction experts from industry and academia to identify methods and techniques used in leading organizations to obtain meaningful feedback from their customers. We performed our work from March 2001 to June 2002 in accordance with generally accepted government auditing standards. Elizabeth G. Mead, Cary B. Russell, David R. Warren, Jeffrey A. Kans, Jack Kriethe, David Schmitt, Patricia Albritton, Brian G. Hackett, Latrealle Lee, and Stanley J. Kostyla also made significant contributions to this report.

Summary: The Defense Logistics Agency (DLA) performs a critical role in supporting America's military forces worldwide by supplying every consumable item--from food to jet fuel--that the military services need to operate. Although customers at the eight locations GAO visited were satisfied with some aspects of routine service, such as delivery time for routine parts and certain contractor service arrangements, customers also raised a number of points of dissatisfaction, particularly with regard to the detrimental impact of DLA's service on their operations. The agency's approach for obtaining customer service feedback has been of limited usefulness because it lacks a systematic integrated approach for obtaining adequate information on customer service problems. Although DLA has initiatives under way to improve its customer service, there are opportunities to enhance these initiatives to provide for an improved customer feedback program.