U.S. Postal Service: Actions Needed to Make Delivery Performance Information More Complete, Useful, and Transparent
Published: 2015-09-30T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: USPS is in the difficult position of balancing cost-cutting actions to address its poor financial situation with efforts to provide prompt, affordable, and reliable mail service. GAO has previously reported that complete, useful, and transparent delivery performance information is essential for USPS and stakeholders to understand USPS's success in achieving this balance.
GAO was asked to review how USPS measures delivery performance and how PRC assesses this information. GAO assessed (1) USPS's measurement of mail delivery performance and related oversight by PRC and (2) USPS's and PRC's reporting of this information. GAO reviewed USPS and PRC delivery performance data for fiscal years 2010-2015, delivery service standards, and measurement system documents, as well as applicable laws and leading practices identified in GAO's prior work.
section_title: What GAO Found
paragraphs: U.S. Postal Service (USPS) measurement of on-time delivery performance has expanded greatly over the past 9 years, but remains incomplete because only 55 percent of market-dominant mail (primarily First-Class Mail, Standard Mail, Periodicals, and Package Services) is included (see fig.). The remaining 45 percent is excluded due to various limitations, such as not having barcodes to enable tracking. Incomplete measurement poses the risk that measures of on-time performance are not representative, since performance may differ for mail included in the measurement, from mail that is not. Complete performance information enables effective management, oversight, and accountability. In addition, the Postal Regulatory Commission (PRC) has not fully assessed why USPS data are not complete and representative. While PRC's annual reports have provided data on the amount of mail included in measurement, they have not fully assessed why this measurement was incomplete or whether USPS actions will make it so. PRC may initiate a public inquiry docket (a type of proceeding) to improve data quality and completeness, but has not done so. Such a proceeding could facilitate evaluating data quality and identifying areas for improvement, as well as actions and time frames to complete improvements.
USPS's and PRC's reports on delivery performance are not as useful as they could be for effective oversight because they do not include sufficient analysis to hold USPS accountable for meeting its statutory mission to provide service in all areas of the nation. USPS's and PRC's reports provide analysis, as legally required. However, this national-level analysis does not facilitate an understanding of results and trends below the national level, such as for USPS's 67 districts, to identify variations and areas where improvements are needed. Further, delivery performance information is not sufficiently transparent or readily available. USPS posts only the most recent quarterly report on its website making it difficult for stakeholders to access trend data. Also, USPS and PRC are not required to provide—and do not report—performance information for rural areas. While several Members of Congress have recently requested studies on rural delivery performance, USPS has stated that such analysis would be costly, even though it could not provide specific cost estimates. Such cost information would be useful for Congress to assess whether developing this information would be appropriate.
section_title: What GAO Recommends
paragraphs: To assist in determining whether to require USPS and PRC to report on delivery performance for rural areas, Congress should direct USPS to provide cost estimates related to providing this information. Further, GAO recommends that USPS and PRC take steps to improve the completeness, analysis, and transparency of delivery performance information. USPS and PRC agreed with the recommendations addressed to them, but disagreed with certain findings on which they are based. GAO believes these findings are valid, as discussed in this report.

section_title: Letter
paragraphs: The U.S. Postal Service (USPS) is in a difficult position of balancing cost- cutting actions needed to address its poor financial situation with efforts to  provide prompt, reliable, and affordable mail service. With insufficient  revenues to cover its expenses and financial obligations, USPS continues  to be in a serious financial crisis. Since fiscal year 2012, USPS has  taken action to reduce its costs, including reducing its career workforce by  about 40,000 employees and consolidating 141 mail processing facilities  in 2012 and 2013. Nonetheless, USPS is charged with meeting its  statutory mission to provide prompt, reliable, and efficient services to all  areas of the country (universal delivery service), including rural areas.  Such a mission requires an extensive network of employees, facilities,  and equipment. Major changes to network operations have the potential  to affect USPS’s ability to provide high-quality, universal delivery service.  We have previously reported that delivery performance information that is  complete, useful, and transparent is essential for enabling USPS and  postal stakeholders—such as the Postal Regulatory Commission (PRC— an independent agency which has regulatory oversight over USPS),  Congress, business mailers,evaluate how well USPS is fulfilling its statutory mission and specific   and the general public—to understand and  statutory requirements for mail delivery.Congress have expressed concern that actions USPS has taken since  2012 to reduce its costs have negatively affected the quality of mail  delivery service, and that reported delivery performance information does  not provide an accurate indication of performance in rural areas. You  asked us to review USPS’s measurement of delivery performance and  PRC’s oversight of this information. This report assesses (1) USPS’s  measurement of mail delivery performance and PRC’s oversight of this  measurement and (2) USPS’s and PRC’s reporting of this information.
See GAO-06-733. decision making, we reviewed the Government Performance and Results  Act of 1993 (GPRA), and the GPRA Modernization Act of 2010  framework for meeting fiscal, management, and performance challenges,  practices used by high-performing agencies, and prior GAO reports.
To determine how USPS measures delivery performance, how PRC  oversees USPS’s measurement, and how USPS and PRC report delivery  performance information, we reviewed USPS’s annual reports to  Congress and PRC, PRC’s annual compliance determinations, USPS and  PRC websites, and other documentation on this measurement and  reporting. We also obtained written responses from USPS and PRC  related to mail delivery performance measurement and reporting and  interviewed USPS, PRC officials, and mailing industry officials. USPS’s  responses contained data on the amount of mail included and excluded  from delivery performance measurement in fiscal years 2010 through the  second quarter of fiscal year 2015. We assessed the reliability of USPS’s  data through a review of related documents, such as written responses  from USPS. We found these data sufficiently reliable for providing a  general description related to the completeness of delivery performance  information. To assess PRC’s oversight of USPS’s measurement of mail  delivery performance, we compared PRC’s efforts to its statutory  authorities and responsibilities related to such oversight. Further details  about our scope and methodology can be found in appendix I.
We conducted this performance audit from October 2014 to September  2015 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: Following enactment of PAEA in 2006, USPS updated its delivery service  standards for market-dominant products, which define the number of days  USPS is to take to deliver the mail in a timely manner. USPS’s delivery  service standards are set forth in federal regulations and differ  depending on the type of mail, the time of day and location at which  USPS receives the mail, and the mail’s final destination. For example,  USPS standards for delivery of 2-day single-piece First-Class Mail require  the mail to be received by a specified cutoff time on the day it is accepted,  which varies depending on geographic location and where the mail is  deposited (e.g., in a collection box, at a post office, or at a mail  processing facility). This mail must then be delivered on the second  regular delivery day (Monday to Saturday) to be considered “on time.”
USPS measures delivery performance against its delivery service  standards. For a given piece of mail, USPS first measures the transit  time—that is, the number of days it takes from the point that the mail is  accepted into USPS’s system until its delivery to a home or business.  Then USPS compares this delivery time against delivery service  standards to determine whether the mail was delivered on time. See  figure 1 for USPS’s delivery performance of single-piece First-Class Mail  from fiscal years 2011 to 2015. The second quarter of fiscal year 2015  experienced a significant decline in on-time delivery performance that  USPS attributes to operational changes enacted in January 2015 coupled  with adverse winter weather. However, performance improved in the next  quarter.
Since 2012, USPS has instituted several initiatives aimed at reducing  expenses in its mail delivery and processing operations and networks as  part of broader efforts to address its fiscal challenges and move toward  financial viability. These initiatives included changing mail delivery service  standards for some types of mail and then consolidating 141 mail  processing facilities in 2012 and 2013. As we reported in September  2014, USPS changes to its delivery service standards increased the  number of days for some mail to be delivered and still be considered on  time. Further, effective January 5, 2015, USPS changed the delivery  service standard for single-piece First-Class Mail sent to a nearby  destination from 1 to 2 days. Table 2 presents these changes for market- dominant mail, which consists of First-Class Mail (e.g., correspondence,  bills, payments, and statements); Standard Mail (mainly advertising);  Periodicals (mainly magazines and local newspapers); and Package  Services (mainly Media/Library Mail and Bound Printed Matter).
To understand how changes in service standards affected expected  transit times for First-Class Mail, we asked USPS to estimate the volumes  of First-Class Mail subject to 1-day, 2-day, and 3-5-day delivery service  standards for fiscal years 2011 through the second quarter of fiscal year  2015—the first quarter after USPS made its most recent changes to  delivery service standards. USPS estimated that the percentage of First- Class Mail volume subject to a 1-day standard decreased from 38 percent  in fiscal year 2011 to 13 percent in the second quarter of fiscal year 2015  (see fig. 2). When on-time delivery of First-Class Mail is redefined from a  1-day standard to a 2-day standard, USPS can take longer to deliver the  mail for it to be considered “on time.”
Based on delivery service standards, USPS sets annual performance  targets for the percentage of mail that is to be delivered on time, and PRC  annually assesses and reports USPS’s performance towards meeting  these targets. PRC has responsibility for assuring delivery performance  data are complete and accurate, and must approve any internal delivery  performance measurement systems (i.e., systems administered by USPS  as opposed to an external contractor).
USPS is subject to legal requirements to create delivery service  standards for market-dominant products, measure delivery performance,  and report the results. Likewise, PRC is subject to legal requirements to  specify how USPS should measure and report delivery performance, as  well as requirements for using these data to provide oversight over USPS  delivery performance. Key requirements for USPS and PRC are  summarized in table 3.
USPS uses two primary methods for measuring delivery performance:
Tracking Barcoded Mail: Since 2011, USPS has measured and  reported delivery performance for most types of market-dominant mail  by using the time it is accepted at postal facilities to “start the clock”  and scans of barcoded mail pieces to “stop the clock” by external,  third-party reporters who receive the mail. This mail transit time is  compared against delivery service standards to determine whether  mail is delivered within the standard and thus considered on time.  Most barcoded mail is tracked through USPS’s Full-Service Intelligent  Mail program, which requires participating mailers that send bulk mail  (i.e., mail entered in bulk quantities such as bills, advertisements, and  magazines) to apply unique Intelligent Mail barcodes to mail pieces  and provide USPS with electronic documentation for each mailing.  Under this program, USPS uses a census approach that aims to  measure all qualifying mail pieces in its mail processing network  rather than a sampling approach. USPS commented that a census- type approach enables it to use the information to better manage day- to-day conditions throughout its network and that such visibility would  not be available through sampling.
Sending Test Mail Pieces: Since 1990, USPS has measured and  reported on-time delivery performance for single-piece First-Class  Mail through the External First-Class Mail measurement system  (EXFC). Under this sampling system, an external contractor arranges  for anonymous droppers to send test mail pieces from street collection  boxes and private office-building lobby chutes to external, third-party  reporters at residences and businesses. As will be discussed in more  detail below, in January 2015, USPS proposed replacing its EXFC  measurement system for single-piece First-Class Mail in favor of a  system based on tracking barcoded mail.
section_title: Delivery Performance Information Has Expanded but Remains Incomplete, and PRC Has Not Fully Assessed Causes of Incomplete Data or the Effectiveness of USPS Actions
subsections: section_title: Measurement of Delivery Performance Has Expanded but Remains Incomplete
paragraphs: USPS’s measurement of on-time delivery performance has expanded  greatly over the past 9 years, but remains incomplete because only about  55 percent of market-dominant mail volume is currently included in  measurement. The remaining 45 percent is not included in  measurement for two main reasons: (1) lack of trackable barcodes or  (2) lack of needed information. USPS told us that it wants to include  virtually all market-dominant mail in delivery performance measurement.  To assess completeness, we determined measurement coverage—the  percentage of mail included in measurement—as well as the various  causes for why mail is not included in measurement and their possible  effect on measured results. To the extent that mail is not included in  measurement, performance data are not complete and may not be  representative. There is not a minimum threshold of mail that is to be  included in measurement for it to be considered representative. In  general, the risk that measurement is not representative is greater if mail  not included in measurement may be systematically different than mail  included in measurement.  In particular, if the unmeasured mail has  different characteristics than the measured mail, and those characteristics  are associated with the likelihood of on-time delivery, then the risk of a  non-representative measurement is greater.
As of the second quarter of fiscal year 2015, USPS measured on-time  delivery performance for about half (55 percent) of market-dominant mail  volume—up from only one-sixth of volume (16 percent) in fiscal year 2006  (see fig. 3). This increase in measurement coverage represents  noteworthy progress by USPS and the mailing industry, which have  devoted management commitment and significant resources to  implement and participate in measurement systems for bulk mail that  comprise most mail volume. Notably, USPS implemented measurement  systems for bulk First-Class Mail, Standard Mail, and Periodicals. The  number of pieces of market-dominant bulk mail included in delivery  performance measurement has increased greatly in recent years—from  96 million pieces in the first quarter of fiscal year 2010 to 14.9 billion  pieces in the second quarter of fiscal year 2015. Thus, the percentage of  market-dominant bulk mail included in measurement increased from less  than one percent in the first quarter of fiscal year 2010 to 48 percent in  the second quarter of fiscal year 2015. Meanwhile, USPS provides  performance measurement that covers virtually all single-piece First- Class Mail through mailing test pieces as part of its long-standing EXFC  system. However, single-piece First-Class Mail comprises a small and  declining percentage of market-dominant mail volume—down from 20  percent in the first quarter of fiscal year 2010 to 14 percent in the second  quarter of fiscal year 2015.
USPS’s measurement coverage has varied by class of mail (see fig. 4).  The percentage of mail included in measurement is greatest for First- Class Mail (both bulk and single-piece mail), followed by Standard Mail,  Periodicals, and market-dominant Package Services (mainly Media  Mail/Library Mail and Bound Printed Matter) and has improved over time  for each class. Most progress began in fiscal year 2011, when mailer  participation increased significantly in Full-Service Intelligent Mail—a  program that enables business mailers to track the progress of their  barcoded mail through USPS’s mail processing system. Meanwhile,  USPS’s EXFC measurement system continues to send test mail pieces to  measure delivery performance of single-piece First-Class Mail.
USPS’s measurement of on-time delivery performance for market- dominant mail remains incomplete because, as noted above, only about  half of bulk mail volume was included in measurement as of the second  quarter of fiscal year 2015. USPS tracks bulk mail using barcodes and  electronic information about the mailing. The main causes for incomplete  measurement of bulk mail can be broadly grouped into two different  reasons: (1) mailers not applying a unique Intelligent Mail barcode to  each mail piece to enable tracking (trackable barcodes)needed information.
In addition, some mailers may not apply trackable barcodes due to the type of mail they  are entering, such as certain types of locally-entered and delivered mail that are not  eligible for barcode-based postage discounts. at a USPS facility. About 4.2 billion pieces of mail (about 45  percent of all exclusions) were excluded in the second quarter of  fiscal year 2015 due to “no start-the-clock.”  2.  No mail piece barcode scan was recorded by USPS’s automation  equipment: This prevents USPS from being able to track the  barcoded mail. About 1.5 billion pieces of mail (16 percent of all  exclusions) were excluded in the second quarter of fiscal year  2015 due to a lack of a barcode scan recorded by USPS  automated equipment.  3.  Inaccuracies in mail preparation: These include deficiencies in  preparing bundles of mail or the quality of barcodes on mail  pieces. About 1.2 billion pieces of mail (13 percent of all  exclusions) were excluded in this quarter due to inaccuracies in  mail preparation.
The percentage of market-dominant mail volume that was eligible for  measurement but excluded for various reasons increased from 21 to 26  percent over the period from the fourth quarter of fiscal year 2013 to the  second quarter of fiscal year 2015. Thus, exclusion of eligible mail from  measurement has become a more important cause for why measurement  data are not complete. USPS officials told us that the increase in  excluded mail volume can be attributed to the overall increase of mail  volume eligible for inclusion in measurement.
USPS told us about several actions it has taken to reduce exclusions,  including steps to improve the scanning of barcoded mail. USPS also  reported that it is collaborating with the mailing industry and working with  individual mailers to improve the quality of mail preparation and  compliance with requirements that can reduce the volume of mail  excluded from measurement. As previously discussed, delivery  performance may be different for mail included in measurement  compared to mail that is not included in measurement. USPS told us that  it has not studied whether on-time delivery performance varies for mail  sent by mailers that do not participate in its measurement programs.  Thus, the effect of non-participation on delivery performance  measurement is unknown. However, available information indicates that  non-participation can affect results for some Standard Mail products— particularly if product-specific results are not weighted to reflect key  characteristics of the mail. Large volume mailers, which are most likely to  apply barcodes and thus have on-time delivery performance measured,  reportedly use additional mailing practices to facilitate the timely delivery  of their mail, such as entering large volumes of advertising mail close to  its final destination. Destination-entered Standard Mail is more likely to be  included in measurement than other “end-to-end” Standard Mail and is  more likely to be delivered on time than end-to-end Standard Mail. For  example, in its latest Annual Compliance Determination Report, PRC  noted that 86 percent of Standard Mail Flats (a Standard Mail product  generally consisting of large flat-sized advertising such as catalogs) were  delivered on time in fiscal year 2014 when they were destination entered  (entered at a postal facility generally closer to the final destination of the  mail), but only 50 to 66 percent were delivered on time when they were  not destination entered. USPS told us that the mail volume and  preparation of destination-entry Standard Mail enable it to be more likely  to be included in measurement as well as be delivered on time.  Destination-entry mail also has sufficient volume and preparation to  enable it to bypass various postal network processing and transportation  (e.g., locally-entered and delivered so it is handled by only one  processing facility), which according to USPS, is the reason this mail is  more likely to be delivered on time.
section_title: USPS Has Proposed Actions to Implement New Measurement Capabilities
paragraphs: In January 2015, USPS proposed replacing its EXFC measurement  system for single-piece First-Class Mail in favor of a system based on  tracking barcoded single-piece mail. USPS stated its proposed system  is intended to incorporate a larger and more representative population of  mail pieces in measurement than EXFC does today. PRC established a  public inquiry docket (a type of proceeding) to review USPS’s proposal,which is still ongoing as of September 21, 2015. In this proceeding, USPS  said it expected the data from its proposed system to provide PRC with  the ability to perform its responsibilities with a high degree of confidence  and to reasonably inform the public regarding the quality of service  provided for market-dominant products.
However, some parties that commented on USPS’s proposal questioned  whether the proposed system would produce representative results. For  example, one concern was that the proposed system would measure mail  deposited in blue collection boxes and at postal retail counters, but not  measure the 38 percent of single-piece First-Class Mail that carriers pick  up from customer mailboxes. Some parties commented that mail picked  up by carriers may arrive at mail processing facilities a day later than mail  deposited into collection boxes, such as situations where carriers return  too late to the office for the mail to be transferred to transportation that  day to a local mail processing facility. Others expressed concern that only  barcoded single-piece mail would be eligible for measurement, which  would exclude stamped mail without a barcode (such as personal  correspondence and greeting cards).
In USPS’s written reply to comments made on its proposal, USPS  responded that critics overstated the difference between handling of mail  left for carrier pickup and collection mail, and that the barcoded mail  measured by its proposed system would provide a reasonable indicator of  performance for all mail collected and transported to the processing  facility. Among other comments, USPS responded that mail processing  for single-piece First-Class Mail is conducted over a range of hours each  day, offering a substantial window of opportunity to accommodate mail  arriving later than normal, including mail that missed the last scheduled  dispatch from facilities where carriers bring stamped mail for forwarding to  processing facilities. USPS further responded that it is not yet feasible for  it to measure delivery performance for single-piece First-Class Mail left for  carrier pickup, but added that it was revising its proposal to measure  stamped and metered mail left at postal retail lobby chutes.
In its June 2015 interim order, PRC commented that because USPS’s  proposal is still in development, PRC lacked sufficient information to  make decisions concerning whether or not the proposed systems will be  suitable for reporting service performance to PRC. PRC added that  given that EXFC appears to have been producing reliable results for a  considerable number of years, PRC cannot approve a new system to  replace EXFC until the new system is similarly operational and verifiable.  PRC directed USPS to plan to run EXFC and the proposed new system in  parallel for a sufficient time to ensure it is operational and verifiable. PRC  explained that test results demonstrating that the EXFC and new system  generate objective and reliable measurements for all affected products  over a period of four consecutive quarters would appear to be an  acceptable demonstration.
On August 25, 2015, USPS filed its statistical design plan for its proposed  new system with the PRC, which explained the sampling methodology  and the methodology for calculating results and their margins of error.  However, USPS had not yet made public some other major aspects of its  proposed new system—such as quality control procedures and internal  controls including methods to address errors in collecting data. Since  USPS has relied on the EXFC system since 1990 to measure delivery  performance for single-piece First-Class Mail, a thorough review of  detailed information on its proposed system will be important not only to  PRC, but to stakeholders including Congress and the mailing industry. In  this regard, PRC’s proceeding to evaluate this new system continues and  the time frame for completion remains open-ended since there is no  statutory deadline and PRC has not established a deadline.
section_title: PRC Has Not Fully Assessed the Causes of USPS’s Incomplete Measurement Data and USPS’s Actions to Improve Data Quality
paragraphs: Although PRC reports have provided data on the amount of mail included  in measurement of delivery performance, these reports have not fully  assessed why these measurements were incomplete or whether USPS  actions will achieve complete performance data. In addition, USPS  officials told us that they have not established a time frame for achieving  complete measurement. PRC uses these performance data to annually  assess USPS’s delivery performance against targets that USPS has  established for on-time delivery. Thus, delivery performance data that are  complete and representative are essential for PRC to correctly determine  whether USPS has met its delivery performance targets. Complete  information is vital for effective management, oversight, and  accountability purposes.
Further, representatives of mailing industry groups and some mailers told  us and commented in PRC proceedings that PRC should become more  involved in issues regarding the quality of measurement data for on-time  delivery performance, including issues regarding the exclusion of mail  from measurement. These representatives provided us with a variety of  suggestions in this regard, such as performing more in-depth and  frequent oversight to ensure USPS measurement is complete. One said  that USPS is still struggling to scan barcoded mail despite joint USPS- mailer efforts over the past decade. Another representative said that  although PRC has oversight of USPS service performance,  measurement, and reporting, there is little consequence to USPS as a  result of not meeting its targets for on-time delivery or for deficiencies in  its measurement and reporting practices. A third said that PRC should  hold USPS accountable for improving measurement data by requiring a  business plan where USPS would lay out the steps it needs to take and  time frames for implementing its initiatives.
PRC’s annual compliance reports have discussed how much mail volume  for each type of mail is included in measurement and when USPS did not  report performance results due to a lack of measurable data. However,  PRC has not fully pursued the main causes for incomplete data (i.e., lack  of trackable barcodes and lack of information causing data to be excluded  from measurement). PRC reports have expressed concern with low levels  of participation for certain types of mail, stating that low levels cause  unreliable measurement. However, these PRC reports have not fully  assessed the effectiveness of USPS actions taken or planned and  associated timeframes with respect to the main causes for incomplete  data. PRC could pursue the causes for incomplete data within its annual  compliance reviews or it may initiate a separate proceeding. Furthermore,  as previously discussed, by law, PRC may initiate a proceeding to  improve the quality, accuracy, or completeness of data that USPS  annually provides to PRC for its annual compliance determination  whenever it appears the data have become significantly inaccurate or can  be significantly improved. PRC officials told us that they have not been  asked by any stakeholder to initiate such a proceeding. Nor has PRC  exercised its option to initiate a proceeding on its own authority to  address issues that impact the completeness of performance data.
PRC and USPS officials told us that they are both opposed to having  PRC initiate a proceeding focused on issues for improving the  completeness of delivery performance measurement for two key reasons.  1.  PRC officials believe that USPS’s delivery performance measures are  generally sufficiently accurate, reliable, and representative for PRC to  meet its legal responsibilities for assessing USPS’s compliance with  service performance standards at the national level. Further, PRC  officials told us that they believe non-measured mail has about the  same on-time performance results as measured mail. However, PRC  and USPS officials told us that neither have compared the  performance of mail included and not included in measurement to  determine if any differences exist. As previously discussed, available  information indicates that non-participation in measurement can affect  reported results for on-time delivery performance. Large volume  mailers, who are most likely to have their mail barcoded and thus  have on-time delivery performance measured, reportedly use  additional mailing practices to facilitate timely delivery, such as  entering large volumes of advertising mail close to its final destination.  Destination-entered advertising mail is more likely to be included in  measurement and is more likely to be delivered on time.  2.  USPS officials stated that a new proceeding to consider data quality  and completeness issues is not necessary because the current  proceeding before the PRC (the performance measurement to replace  EXFC) provides a public forum for consideration of the quality of  service performance data, as well as mail excluded from  measurement. However, according to publicly available documents in  the current proceeding, PRC has not explored issues of delivery  performance measurement data for bulk mail that are excluded from  USPS’s current measurement systems, the multiple causes for these  exclusions, and USPS actions under way and planned to address the  causes. The proceeding also has not thoroughly explored mailers’  concerns regarding data exclusions, such as exclusion rules and  mailer views regarding time frames for making progress on reducing  exclusions.
A PRC proceeding that focuses solely on issues of data quality and  completeness—particularly the problem of data exclusions—may facilitate  these issues receiving the fullest attention and making more rapid  progress by USPS and the mailing industry toward achieving more  complete measurement. As previously noted, while USPS has made  progress toward achieving completeness since 2006—as illustrated by  figure 3 earlier in this report—45 percent of market-dominant mail is still  not measured. Performance information is sufficiently complete when it  has the coverage to enable representative measurement of the  percentage of mail delivered on time. While there is not a minimum  threshold of mail that is to be included in measurement for it to be  representative, the risk that measurement is not representative increases  as more mail is not included in measurement because on-time delivery  performance may be different for mail that is included in measurement  from mail that is not included. Therefore, having a proceeding solely  focusing on data quality and completeness could give USPS and postal  stakeholders such as PRC, Congress, business mailers, and the general  public the opportunity to conduct an in-depth evaluation of the quality of  delivery performance data, identify practical opportunities to improve data  quality, and establish actions and time frames for making progress.  Having such a proceeding also could help PRC develop a better  understanding of issues regarding the quality of delivery performance  data and thereby be in a better position to conduct ongoing oversight of  data quality and its annual compliance determination.
section_title: Reported Delivery Performance Information Is Not Sufficiently Useful or Transparent for Oversight of Delivery Performance in all Areas
paragraphs: USPS and PRC reports on delivery performance are not as useful as they  could be for effective oversight.  USPS and PRC annual compliance  reports provide delivery performance analysis, as legally required. This  information is reported at the national level.  This analysis, however, does  not facilitate an understanding of results and trends below the national  level, such as for USPS’s 67 districts, to identify variations and areas  where improvements in performance may be needed. USPS and PRC  annual and quarterly reports on delivery performance information are not  as useful for other oversight purposes or management and congressional  decision making. For example, these reports do not include sufficient  analysis to hold USPS accountable for meeting its statutory mission to  provide prompt, reliable, and efficient services in all areas of the nation  and regular postal services to rural areas. Further, delivery performance  information is not sufficiently transparent as it is not readily available on  respective USPS and PRC websites. Thus, it is difficult for effective  oversight and for stakeholders to understand trends and develop analysis  of USPS performance information.
subsections: section_title: Delivery Performance Information Is Not Reported in a Useful or Transparent Manner to Hold USPS Accountable for Meeting Its Mission
paragraphs: We have reported that ensuring information is useful to assist  management and congressional decision making is key to the principles  embodied in GPRA and the GPRA Modernization Act of 2010 framework  for meeting fiscal, management, and performance challenges. USPS  and PRC reports, however, provided little analysis to facilitate an  understanding of results and trends below the national level. USPS and  PRC websites do provide annual and quarterly delivery performance  results on the national level and for each of USPS’s 7 areas and 67  districts. In addition, PRC provided annual delivery performance trend  data at the national level in its annual compliance determinations covering  fiscal years 2013 and 2014.however, are not sufficiently useful for determining variations in delivery  performance across the nation or determining whether performance has  improved in areas where performance has not met service standards or  targets. National averages aggregate the mail delivery performance of  different parts of the country into an average for the entire nation. Thus,  on-time delivery performance in one section of the country may be  masked by on-time delivery performance in another section of the  country. A national average alone does not enable stakeholders to  understand if certain areas of the country are experiencing poor delivery  performance.
Trend data solely at the national level,  To better understand the range and variations in delivery performance  across the nation, we analyzed trends in quarterly delivery performance  at the district level. Our analysis showed how national data can mask  wide variations in performance by various districts over time. For  example, we analyzed quarterly performance for single-piece First-Class  Mail with a 3-to-5-day delivery service standard for each of the 64 postal  districts in the contiguous 48 states and the District of Columbia for the  second quarter of fiscal years 2013 to 2015. For the second quarter of  fiscal year 2015, none of the districts met that quarter’s performance  target of 95 percent of mail delivered on time. Performance for that  quarter ranged from 44 percent to 80 percent (see fig. 5). However, when  analyzing the second quarter of the previous 2 fiscal years, of the 10  districts with the lowest scores in the second quarter of fiscal year 2015, 9  were below the national average in fiscal year 2014, and all 10 were  lower than the national average in fiscal year 2013, but to a much lesser  degree. Of the 10 districts with the highest scores in the second quarter of  fiscal year 2015, 8 were above the national average in the second quarter  of fiscal years 2014 and 2015.
In addition, USPS’s reporting of delivery performance information is not  sufficiently transparent. To be considered transparent, the criteria we  identified suggest that delivery performance information is to be reported  in a manner that is easily accessible and readily available. USPS,  however, posts only its most recent quarterly report of area and district- level data on its public website.request numerous files from USPS to compile data necessary for  understanding performance trends, such as whether on-time delivery is   As a result, stakeholders would have to  improving or getting worse. USPS told us that its reporting of delivery  service information meets statutory requirements, and that it is not  required to maintain quarterly trend data for delivery performance on its  website. However, USPS can elect to maintain quarterly trend data on its  website. A large mailer association we spoke with stated that USPS  should be so transparent that everyone understands general performance  and any factors contributing to good and poor performance.
Similar to USPS, PRC’s reporting of delivery performance information is  not readily available to stakeholders. While PRC also posts delivery  performance information provided by USPS on its public website,  stakeholders would have to find numerous files in multiple locations on its  website to compile data necessary for understanding performance trends,  such as whether on-time delivery is improving or getting worse. In  addition, PRC’s reports are not easily accessible. PRC has reported its  annual assessment of USPS’s delivery performance in fiscal year 2014 in  two reports that are filed on its website at different times and at different  links, while USPS’s quarterly data are posted at another link on PRC’s  website. The lack of easily accessible and readily available performance  information on USPS’s and PRC’s part impedes the ability of Congress,  mailers, and customers to review and hold USPS accountable for its  performance and to use the information to develop realistic expectations  for when their mail will be delivered.
section_title: Rural Delivery Performance Information Is Not Required to Be Reported
paragraphs: USPS and PRC are not required to report—and do not report—delivery  information for rural and non-rural areas, thus limiting effective oversight  in these areas. USPS and PRC officials told us that they do not provide  information or analysis to assess delivery performance specifically for  rural areas because they are not legally required to do so. Without data  on rural delivery performance, Congress cannot determine the extent  delivery performance is timely in rural versus non-rural areas, and neither  USPS nor PRC can prove or disprove any perceptions that rural areas  may be affected differently than non-rural areas.
Several Members of Congress and others have raised questions about  whether delivery performance in rural areas has been negatively affected  by changes USPS has implemented since fiscal year 2012 to reduce its  expenses. For example, according to the National Newspaper  Association (NNA), community newspapers have been negatively  affected since USPS consolidated some postal facilities. Further,  problems have emerged when newspapers, often in rural areas, had to  be delivered outside of the local area and experienced a decline in  service. NNA has requested that PRC gather information about the data  that could be produced about rural mail to identify the sources of delivery  problems, such as manual processing, increased travel distances, or  inefficient processing plants. NNA has argued that “the possibility that  what ails NNA newspapers also ails rural mail in general is more than a  random guess.” In May 2015, two Members concerned about the lack of  digital tracking in rural areas requested a PRC study on the feasibility of  reporting on rural mail delivery performance. Other congressional  requests for rural delivery performance information are also pending. For  example, in a recent Senate report, the Senate Appropriations Committee  directed USPS to take steps related to reporting delivery performance in  rural areas. In July 2015, the Senate report accompanying the Senate  Financial Services and General Government Appropriations Bill, 2016  directed USPS and PRC to report mail delivery performance to  specifically include mail delivery from rural towns to other rural towns;  from rural towns to urban areas; and from urban areas to rural towns.  The Committee requested the methodology used to develop this  information within 60 days of enactment of the Act with a subsequent  report due by March 1, 2016.
USPS has not reported data on on-time delivery performance based on a  rural or non-rural distinction. USPS officials told us that no overall  assessment of rural delivery service, separate and apart from  urban/suburban delivery, has been undertaken since PAEA required  delivery performance measurement, reporting, and assessment. USPS  officials added that its delivery performance data provide a basis for  internal diagnosis and assessment of operations and service, and satisfy  USPS’s reporting obligations to PRC. USPS officials noted that its reports  are generated at the national, area, and district level for these purposes,  but are not routinely further disaggregated on the basis of whether  particular districts or ZIP Codes are rural, suburban or urban in nature.  On-time delivery performance information at the district level cannot  inform stakeholders on delivery performance in rural areas since each of  USPS’s 64 districts in the continental United States contains at least one  core area with a population over 10,000 and thus is not entirely rural.
USPS officials told us that USPS’s service performance measurement  systems do not differentiate between rural and urban locations and that it  may be cost prohibitive to attempt to measure performance of mail pieces  in rural areas using an external data system.
However, in response to the recent congressional request for PRC to  report on rural mail delivery performance, USPS told us that it has begun  collaborating with the technical staff at PRC to determine how  measurement may account for rural origin and destination points and that  its new, proposed internal service performance measurement plan might  provide greater insight on service performance measurement specific to  rural areas, assuming USPS and PRC can arrive at a reasonable  definition of “rural” origin and destination points. At this time, however,  USPS officials added that USPS was at an exploratory stage of the  analysis and were not able to offer definitive conclusions on the feasibility  of adding this feature to USPS’s measurement plans.
PRC officials told us that they are currently working with USPS to  determine how they will respond to the congressional request for rural  delivery performance information. PRC officials also told us that PRC has  limited its previous assessments regarding whether USPS met its delivery  service standards for market-dominant types of mail to national results  and has not conducted any rural-level analysis. PRC officials told us that  PRC does not play a direct role (e.g., either annually or quarterly) in  monitoring or reporting on USPS’s universal delivery service obligation  (aside from annually estimating the cost of universal postal service),  noting that PRC is not legally required to do so, nor has PRC been  directed by Congress to play this role. PRC officials added that PRC has  not considered requiring USPS to report quarterly and annual information  on delivery speed and reliability in urban versus rural areas, because 1)  PRC has not been specifically mandated by statute to require USPS to  provide delivery service performance information separately for rural and  urban areas and 2) in PRC’s previous assessments, Congress has not  provided specific direction requiring USPS to implement such  measurement and reporting.
As noted previously, USPS officials told us that the costs of additional  requirements for USPS to collect and report urban and rural delivery  performance information through existing measurement systems would  likely greatly outweigh the benefits. However, USPS and PRC were not  able to provide specific cost estimates related to having USPS measure  and report on delivery performance in rural and urban areas. We asked  USPS for this information, but it did not provide such cost information,  with USPS officials explaining that there is no clear definition or defined  approach to measure what should be considered rural. USPS officials  also told us that the cost would depend on the specificity of the data, such  as whether there would be national-level results for urban and rural areas  or detailed geographic breakdowns. We also asked PRC about the costs  of providing delivery performance information in rural and urban areas.  PRC responded that it has the authority to specify requirements for  USPS’s delivery performance measurement, but that when considering  reporting requirements for USPS, it is to give consideration to  unnecessary or unwarranted administrative effort and expense by USPS.  On this matter, PRC officials said that they do not know what the costs  might be for USPS to collect data on delivery performance in rural and  urban areas. Neither of the congressional directives mentioned above  regarding studying delivery performance in rural areas directly address  the costs associated with requiring rural delivery performance information.  Without cost estimates, Congress may not have all the information it  needs to understand the full implications of requiring data on delivery  services in rural and non-rural areas.
section_title: Conclusions
paragraphs: Quality delivery performance information is needed for USPS and postal  stakeholders such as PRC, Congress, business mailers, and the general  public to develop useful analysis that can help oversee or assess the  balance between USPS’s cost-cutting to address its poor financial  situation while maintaining affordable postal rates and providing timely,  universal delivery service. Thus, it is important for both USPS and PRC to  report delivery performance information in a sufficiently complete,  transparent, and useful manner.
Although USPS has made progress since PAEA was enacted in 2006, its  delivery performance information is not complete, and it is unclear when  USPS will achieve its goal of measuring on-time delivery for nearly all  market-dominant mail volume. USPS measured on-time delivery for only  55 percent of market-dominant mail volume in the second quarter of fiscal  year 2015. As a result, data may not be representative because  performance may be different for mail not included in measurement.  Although PRC’s reports provide data on the amount of mail included in  measurement, they have neither fully assessed the reasons why these  measurements are incomplete, nor specified what actions USPS needs to  take and the related time frames needed to achieve complete  performance measurement. PRC may initiate proceedings to improve the  completeness and quality of delivery performance data, but it has not  exercised this option. Although USPS and PRC are opposed to such a  proceeding, we believe that a PRC proceeding that focuses on issues of  data completeness—particularly the problem of excluding mail due to a  lack of information—could facilitate more rapid progress by USPS and the  mailing industry toward complete measurement.
USPS and PRC annual and quarterly reports on delivery performance  information are not as useful for oversight purposes beyond the annual  compliance assessments because they do not include sufficient analysis  that would facilitate holding USPS accountable for meeting its statutory  mission to provide prompt, reliable, and efficient services in all areas of  the nation, including rural areas. For example, neither USPS nor PRC  reports trend data below the national level for all of USPS’s 67 districts to  indicate whether performance is improving or getting worse in different  parts of the nation. Further, delivery performance information is not  sufficiently transparent as it is not readily available or easily accessible on  either USPS’s or PRC’s website. Also, postal stakeholders—such as  PRC, Congress, business mailers, and the general public—cannot  determine whether delivery performance is a problem in rural areas  because USPS and PRC are not required to report delivery performance  information separately for rural versus non-rural areas. USPS believes  that such an analysis would be costly, even though it does not know how  much it would actually cost. Such cost information would be useful for  Congress to have in order to assess whether developing this information  would be appropriate. In addition, USPS and PRC are in the process of  responding to a recent congressional request to determine the feasibility  of reporting on rural mail delivery performance, which could facilitate  determining the associated costs.
section_title: Matter for Congressional Consideration
paragraphs: To assist in determining whether to require USPS and PRC to report on  delivery performance for rural and non-rural areas, Congress should  direct USPS to provide cost estimates related to providing this  information.
section_title: Recommendations for Executive Action
paragraphs: To improve the completeness of USPS delivery performance information,  we recommend that the Acting Chairman of PRC and the other PRC  Commissioners exercise PRC’s statutory authority to hold a public  proceeding involving USPS, the mailing industry, and interested parties to  address how USPS can improve the completeness of USPS’s delivery  performance information.
To improve the usefulness and transparency of USPS’s and PRC’s  reporting of delivery performance information, we recommend that:
The Postmaster General provide additional and readily available  delivery performance information, such as trend data for on-time  delivery performance for all 67 postal districts.
The Acting Chairman of PRC and the other PRC Commissioners  provide readily available data and additional analysis of USPS’s  delivery performance information so that stakeholders can better  understand trends and variations in mail delivery performance.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to USPS and PRC for review and  comment.  USPS and PRC provided written responses, which are  reproduced, respectively, in appendixes II and III of this report. PRC and  USPS agreed with the recommendations addressed to them. Specifically,  PRC agreed to hold a proceeding to address how USPS can improve the  completeness of USPS’s delivery performance—after, as we reported,  initially indicating it was opposed to such a proceeding. Although not the  addressee of this recommendation, USPS disagreed with it stating that its  measurement systems conform to the Office of Management and  Budget’s standards and guidelines for statistical surveys, and that it  employs a contractor with long-standing expertise in developing  statistically valid and reliable systems.  However, we found that key data  quality issues involve the lack of completeness of census-type  measurement and the associated risk of non-sampling error—issues that  are separate from matters of statistical design. Further, USPS said its  continuing collaboration with the mailing industry is more likely to  stimulate industry cooperation and buy-in rather than lengthy, time- consuming proceedings before the PRC.  We continue to believe that a  new PRC proceeding on data quality would add value to its continuing  collaboration with USPS and the mailing industry.  Our report notes that  representatives of mailing industry groups and some mailers told us, and  commented in PRC proceedings, that PRC should become more involved  in issues of the quality of measurement data for on-time delivery  performance, including issues regarding the exclusion of mail from  measurement.
Both USPS and PRC agreed with our recommendations to improve the  usefulness and transparency of delivery performance information that  they report.  USPS acknowledged that the delivery service performance  data it reports on its website lacks the granularity of the reports it publicly  files with PRC, and does not serve the purposes of in-depth  congressional oversight.  USPS added that although it is not clear that  typical household mailers would use or find value in that level of data  granularity, it will pursue establishing a distinct portal for public access to  delivery service performance reports, in an effort to be more transparent.   Likewise, PRC agreed that information it receives and produces regarding  performance measures for USPS can be better organized, and it has  updated its website in response to our recommendation.  We are  encouraged by USPS’s and PRC’s willingness to adopt these   the Congress’s  recommendations since, as we have recently reported,and the public’s confidence in the quality of performance information that  federal agencies are using to assess and achieve results requires that  information be publicly reported in a clear and readily accessible way.
Although both agencies agreed with the recommendations addressed to  them, they disagreed with certain findings, conclusions, and the  supporting analytical basis used in this report. Key among the  disagreements were our treatment of the completeness of USPS’s data,  the appropriateness of the criteria we used for our assessment of PRC’s  oversight and analysis, and the usefulness of USPS’s and PRC’s  reporting on delivery performance.
Regarding our treatment of data completeness, USPS said it understood  in the abstract the basis for our finding that delivery performance may  differ for mail included in measurement than mail that is not measured.  However, it disagreed that this is the case in practice. Specifically, USPS  stated that mail pieces that are included in performance measurement  should have virtually the same performance results as mail not included  in measurement. Additionally, PRC stated that data reliability has  markedly improved as a result of PRC’s directives to USPS regarding  measurement systems. As discussed in our report, although the  completeness of measurement has improved over the past 9 years, 45  percent of market-dominant mail is still not included in measurement.  Also, we identified a number of reasons to be concerned that delivery  performance may be different for mail that is included in measurement  than mail not included in measurement. For example, large-volume  mailers, who are most likely to apply barcodes and thus have on-time  delivery performance measured, use additional mailing practices to  facilitate the timely delivery of their mail, such as entering large volumes  of advertising mail close to its final destination. In addition, destination- entry mail has sufficient volume and preparation to enable it to bypass  various postal network processing and transportation (e.g., locally entered  and delivered so it is handled by only one processing facility)—a strategy  that, according to USPS, is the reason this mail is more likely to be  delivered on time.  As we reported, USPS has set a goal of including  virtually all market-dominant mail in measurement, using a census-type  approach, and continues to strive for including more mail volume in  measurement. Further, USPS raised concern that increasing the  proportion of mail included in measurement data may come at potential  significant costs, which we believe is a topic that could be further explored  in the recommended proceeding.
USPS disagreed with the specific example given in our report that  destination-entered Standard Mail is more likely to be included in  measurement than other “end-to-end” Standard Mail and is more likely to  be delivered on time. This example suggests that results for Standard  Mail as a whole are higher than they would be if all Standard Mail were  included in measurement. USPS said that when compiling the national  on-time delivery percentage for all Standard Mail, it weights results for  measured mail pieces by shape and entry type so they are compiled in  proportion to their prevalence in the entire population of Standard Mail.   However, it does not appear that USPS applies weighting procedures  when compiling results for individual Standard Mail products—such as  Standard Mail Flats and Standard Mail Letters that have significant  proportions of both destination-entry and end-to-end mail.  Therefore, we  have clarified our report to state that available information indicates that  non-participation can affect results for some Standard Mail products,  particularly if product-specific results are not weighted to reflect key  characteristics of the mail.
In addition, USPS and PRC made various comments on the importance  of the statistical properties of USPS systems that measure on-time  delivery performance.   USPS said it employs a firm with long-standing  expertise in developing measurement systems that are statistically valid  and reliable, and provided a letter from this contractor that the  measurement systems are designed in a manner to be statistically valid  and representative.  PRC said it reviews USPS data using statistical  principles that determine whether service performance data are sufficient  and the results are meaningful.  Specifically, PRC said sampling fractions,  confidence intervals, and margins of error are the primary factors it uses  to determine whether data are accurate and reliable.  We agree statistical  considerations should inform the assessment of data collected through  sampling. However, we also note that such statistical principles are not  relevant to evaluate the quality of incomplete data collected using census- type measurement, which is the case for most types of market-dominant  mail.  An error created when non-measured mail has different on-time  delivery performance than measured mail is a “non-sampling error”—as  opposed to a “sampling error” that is associated with measurement based  on a random sample. Non-sampling errors can affect results, regardless  of how valid the statistical design of USPS’s measurement systems may  be.  As previously discussed, results based on incomplete data can be  affected when the measurement process disproportionately includes mail  that is more likely to be delivered on time.
PRC also was critical of our focus on data completeness, stating that it is  not a meaningful statistical measure and that PRC has not concluded that  the percentage of mail in measurement should be the primary  determinant of accurate, reliable, or representative measurement data.   USPS stated that our report does not specify what an appropriate level of  measurement may be.
However, PRC has not defined what an appropriate percentage of mail in  measurement would be for measuring on-time delivery based on census- type measurement, while USPS has set a goal of including virtually all  mail volume in measurement.  For most types of market-dominant mail,  measuring on-time delivery performance involves census-type  measurement as well as measurement based on sampling.  For the mail  that is included in the measurement based on a census-type approach, to  assess non-sampling error would require determining whether the mail  not included in measurement systematically differed from the mail  included in the measurement, particularly regarding characteristics  associated with on-time delivery.
Regarding the appropriateness of the criteria we used for our assessment  of PRC’s oversight and analysis, PRC stated that our assessment was  based on GAO-created criteria rather than the statutory requirements in  PAEA. We agree that our assessment was not intended to determine  PRC’s compliance with its statutory requirements.  Rather, our review  used criteria that are appropriate for assessing an organization’s  practices for reporting delivery performance information that would be  useful for management and congressional decision-making. As noted in  our report, the criteria we used to assess USPS’s and PRC’s  measurement and reporting of delivery performance information are  based on current laws—including PAEA—and regulations, as well as  previously identified practices used by high-performing agencies, and  prior GAO reports. Specifically, our criteria are a result of reviewing  delivery performance measurement and reporting provisions applicable to  USPS and PRC in PAEA, and PRC regulations, which we summarized in  table 3. In addition, we believe that certain government principles can  help inform congressional and executive branch decision-making to  address challenges.  For example, USPS should disclose more  information about the accuracy and validity of its performance data and  actions to address limitations to the data. Our prior work has found that  without useful performance information, it is difficult to monitor agencies’  progress toward critical goals.
PRC also disagreed with a statement in our draft report that its reports  have not assessed why USPS’s delivery performance measurements  were incomplete nor specified what actions USPS needs to take to  achieve complete performance-measurement data. PRC said it has  assessed the primary reasons measured mail may be inaccurate,  unreliable, or not representative of nationwide performance, including  data not in Full-Service Intelligent Mail, uncategorized mail, invalid data,  and low district-level volumes. PRC said its reports have regularly  directed USPS to improve data reliability and accuracy by increasing  participation in Full-Service Intelligent Mail, increasing measured volumes  for mail product categories in certain districts, and increasing the number  of districts providing results.
We agree that PRC reports have addressed some issues related to the  quality of delivery performance data, such as providing data on the  amount of mail included in delivery performance measurement and  expressing concern with low levels of participation for certain types of  mail. However, as we discuss in our report, PRC has not fully assessed  why these measurements were incomplete, whether USPS actions will  achieve complete performance data, why lack of participation remains a  significant issue, and whether there are practical opportunities to make  progress. Further, recent PRC reports have not assessed what have  become the primary causes for excluding mail pieces from measurement,  including no “start-the-clock” information, no mail piece barcode scan  recorded by USPS automation equipment, and inaccuracies in mail  preparation. Thus, we continue to believe that PRC has opportunities to  improve its oversight and encourage PRC and all stakeholders to explore  these causes in its forthcoming proceeding.
Regarding the usefulness of reporting, PRC and USPS disagreed with  our characterization that USPS’s and PRC’s reports are not sufficiently  useful for effective oversight.  PRC objected to the implication that it is not  fully successful in meeting its oversight responsibilities, and added that it  has provided strong oversight in achieving the transparency and  accountability required by Congress and that its reports are useful.  While  we recognize that PRC is not statutorily required to assess USPS’s  performance in providing mail to all parts of the country—including rural  areas—USPS is still responsible for adhering to these requirements and  no other oversight agency exists to hold USPS accountable to these  requirements. Given the broad scope of recent changes in postal  operations, network consolidations, and service standard changes,  Members of Congress and other postal stakeholders have raised  concerns about the impact of these changes on delivery performance.  Thus, effective oversight is even more critical to ensure that any delivery  performance problems are promptly identified and addressed.
USPS and PRC also provided technical comments that we incorporated  as appropriate.
As agreed with your offices, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report date. At that time, we will send copies to the appropriate  congressional committees, the Postmaster General, the Acting Chairman  of the Postal Regulatory Commission (PRC), the other PRC  Commissioners, and other interested parties. In addition, the report will be  available at no charge on the GAO website at http://www.gao.gov.
If you or your staff has any questions about this report, please contact me  at (202) 512-2834 or rectanusl@gao.gov. Contact points for our Offices of  Congressional Relations and Public Affairs may be found on the last page  of this report. GAO staff making key contributions to this report are listed  in appendix IV.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: This report assesses (1) the U.S. Postal Service’s (USPS) measurement  of mail delivery performance and the Postal Regulatory Commission’s  (PRC) oversight of this measurement and (2) USPS’s and PRC’s  reporting of this information.
To conduct this work, we assessed whether USPS’s measurement of its  delivery performance is complete and whether USPS’s and PRC’s  reporting on this performance is useful and transparent. To make our  assessments, we compared USPS’s and PRC’s measurement and  reporting efforts to specific elements associated with these criteria. We  originally developed these criteria for a 2006 report that assessed USPS’s  delivery service standards, measures, and reporting. In developing those  criteria, we identified applicable laws related to USPS’s mission,  ratemaking, and reporting, and practices used by high-performing  organizations related to delivery service standards, measurement, and  reporting, including practices identified through our past work. For this  review, as table 4 below illustrates, we adapted and updated each criteria  identified in the 2006 report. We reviewed current laws, previously  identified practices used by high-performing agencies, and prior GAO  reports to identify specific, observable elements associated with each  criteria, in order to make a more direct assessment on the extent delivery  performance information is complete, useful, and transparent. For  example, we reviewed provisions in the Postal Accountability and  Enhancement Act (PAEA) and implementing PRC regulationsestablished the legal framework for measurement of mail delivery   that  performance, PRC’s oversight of this measurement, and reporting of this  information. To identify practices for reporting delivery performance  information that would be useful for management and congressional  decision making, we reviewed the Government Performance and Results  Act of 1993 (GPRA), and the GPRA Modernization Act of 2010  framework for meeting fiscal, management, and performance challenges,  practices used by high-performing agencies, and prior GAO reports.
To assess delivery performance measurement, we reviewed  documentation of mail delivery performance, the measurement systems  used to develop this information, and limitations of these systems. We  also reviewed USPS’s annual reports to Congress and PRC, PRC’s  annual compliance determinations, Mailers’ Technical Advisory  Committeemeasurement systems and the data USPS collects. In addition, we  reviewed relevant documentation regarding USPS’s proposal to replace  its External First-Class Mail measurement system (EXFC), including  USPS’s proposal, stakeholder comments on the proposal, and USPS’s  reply responses to stakeholder comments. Between December 2014 and  June 2015, we received written responses and data from USPS and PRC  related to mail delivery performance measurement and associated  limitations and interviewed USPS and PRC officials. USPS’s responses  contained data on the amount of mail ineligible for delivery performance  measurement and excluded from delivery performance measurement in  fiscal years 2010 through the second quarter of fiscal year 2015. We  assessed the reliability of USPS’s data through a review of related  documents, such as written responses from USPS. We found these data  sufficiently reliable for providing a general description related to the  completeness of delivery performance information. To assess PRC’s  oversight of delivery performance information, we reviewed PRC’s annual  compliance determinations and other reports, obtained written responses  from PRC and USPS, and interviewed PRC and USPS officials. We also  interviewed representatives of mailing industry groups and business  mailers with expertise on delivery performance measurement and postal  issues to discuss the completeness of delivery performance information  reported by USPS and PRC’s assessment of this information. We used  our professional judgment to select these representatives; thus, the  responses we received from them are not generalizable to the entire  mailing industry. We also reviewed laws, regulations, and PRC orders  and determinations to identify any guidance or requirements for USPS  and PRC related to the quality of delivery performance information.   presentations, and other documentation on USPS’s current  To assess reported delivery performance information, we reviewed the  mail delivery performance information reported in USPS annual reports to  Congress, PRC annual compliance determinations and other reports, and  on the USPS and PRC websites. We assessed the usefulness of the  reported information to provide oversight over how effectively USPS  fulfills its statutory mission to provide prompt, reliable, and efficient  services to all areas of the country (universal delivery service), and a  maximum degree of effective service in rural areas. We also interviewed  representatives of mailing industry groups and business mailers with  expertise on delivery performance measurement and postal issues to  discuss the usefulness of delivery performance information reported by  USPS and PRC. We used our professional judgment to select these  representatives; thus the responses we received from them are not  generalizable to the entire mailing industry. To determine the extent that  the delivery performance information is transparent, we reviewed delivery  performance information USPS and PRC disclose on their websites to  assess the extent to which it is easily accessible and readily available.  We also reviewed laws and statutory regulations to identify any  requirements related to reporting delivery performance information in a  transparent manner.
We conducted this performance audit from October 2014 to September  2015 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
section_title: Appendix II: Comments from the U.S. Postal Service
section_title: Appendix III: Comments from the Postal Regulatory Commission
section_title: Appendix IV: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the individual named above, key contributors to this report  were Teresa Anderson (Assistant Director); Samer Abbas; Kenneth John;  Thanh Lu; Malika Rice; Amy Rosewarne; Kelly Rubin; and Crystal Wesco.