Telecommunications: Current Broadband Measures Have Limitations and New Measures Are Promising but Need Improvement
Published: 2009-10-09T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Broadband Data Improvement Act, enacted in 2008, established a variety of initiatives intended to improve the quality of state and federal data on broadband (i.e., high-speed Internet) services and promote the deployment (the building of infrastructure over which broadband services can be provided) of affordable broadband services to all parts of the nation. The act required GAO to conduct a study to consider and evaluate additional broadband metrics or standards. This mandated report addresses (1) the measures generally available to consumers, industry, government and others, and (2) the limitations, if any, of the measures and how they could be supplemented or improved. To identify and evaluate the measures, GAO conducted a review of literature and related laws and interviewed and reviewed related documentation from stakeholder groups.
section_title: What GAO Found
paragraphs: Multiple measures are generally available to consumers, industry, and government to assess broadband performance. Consumers can generally access measures of availability, price, advertised speed, and actual delivered speed from providers and third parties to compare services. Industry and government also have access to some measures that enable comparisons across segments of the United States to inform policy and guide investment. For example, the Federal Communications Commission's (FCC) data from its semiannual reporting requirement for providers are the primary source for comparing the availability of and subscribers to broadband. Through a literature review and interviews with stakeholders, GAO focused on 10 measures that can be used to make international comparisons of broadband service to inform policy. Eight were composite indexes that are generally used to account for factors such as demographic and economic differences among countries, which, according to stakeholders, can affect broadband deployment and penetration (the number or percentage of subscribers per capita or per household). Through available documentation and discussions with stakeholders, GAO found that current measures have limitations, views were mixed on potential alternatives, and ongoing efforts need improvement: (1) According to some stakeholders, the lack of comprehensive measures from the government to compare price, actual delivered speeds, and service reliability data from providers is a limitation for consumers. FCC has open proceedings on requiring providers to report such information, but there was no consensus among stakeholders on the need for additional reporting requirements and measures. (2) Stakeholders told GAO that FCC's semiannual data collection from providers does not include information on availability, price, or actual delivered speeds, which limits the ability to make comparisons across the country and inform policy or investment decisions. Stakeholders generally agreed that the Department of Commerce's effort to develop a national broadband inventory map through its State Broadband Data and Development Grant Program would address some gaps and provide detailed data on availability, subscribership, and actual delivered speeds, but the department did not provide guidance to grantees on calculating actual delivered speeds or specific standards to verify the data collected. This could result in inconsistent data and limit the effectiveness of the effort. GAO has previously reported that consistency and data verification are important for reducing the risk of producing inaccurate data. (3) Finally, the measures used for international broadband comparisons have limitations for a variety of reasons, including socioeconomic differences that make the comparisons difficult. Despite the concerns, stakeholders found the measures useful to help inform policy. Stakeholders generally supported FCC's efforts to develop international comparisons because the comparisons will be at a local level within each country, and could provide more relevant information.

section_title: Letter
paragraphs: Universal access to the Internet via broadband technologies—commonly  referred to as broadband Internet access—is commonly viewed as a vital  public infrastructure and a key driver of economic growth. For example,  broadband technology makes it possible for patients to receive medical  attention from specialists hundreds of miles away, students to access  information not available from their local libraries, school systems to use  one teacher to teach students in multiple schools, and small businesses to  advertise and market their products and services to attract customers.  The Broadband Data Improvement Act, enacted in 2008, established a  variety of initiatives intended to improve the quality of state and federal  data on the availability and quality of broadband services, and promote the  deployment of affordable broadband services to all parts of the nation.  The act required GAO to conduct a study to consider and evaluate  broadband metrics or standards that may be used by industry and the  federal government to provide consumers with better information about  the cost and capability of their broadband connection, and compare the  deployment (the building of infrastructure over which broadband services  can be provided) and penetration of broadband across the United States  and among other countries.
Various policy makers believe that disparities in broadband access across  the United States, including rural areas, could have adverse consequences  for unserved or underserved populations. In the Telecommunications Act  of 1996, the Federal Communications Commission (FCC) and state  regulatory commissions were directed to encourage the deployment of  advanced telecommunications capability, including broadband, to all  Americans. As a result, FCC collects broadband deployment data from  the private sector semiannually using the FCC Form 477, a standardized  industry census. The Broadband Data Improvement Act cited a need for  improved broadband data to help the government better understand the  extent of broadband deployment, develop and maintain appropriate  broadband policies, and direct limited financial resources. In addition,  some broadband advocates have argued that broadband is an important  international indicator of economic strength; thus the United States seeks  additional measures of the nation’s progress compared with that of other  countries. To respond to the requirement in the Broadband Data  Improvement Act that GAO evaluate additional broadband metrics or  standards, this report addresses the following questions:  (1) What measures are generally available to consumers, industry,  government, and other stakeholders to assess broadband performance?  (2) What, if any, are the limitations of these measures, and how can the  measures be supplemented or improved?
To identify the broadband performance measures available, we conducted  a review of the literature and related laws and interviewed representatives  and reviewed related documentation from the following stakeholder  groups: academicians and think tanks, broadband providers, consumer  advocacy groups, federal and state agencies and public/private  partnerships, international organizations, and trade and industry groups.  To evaluate the limitations, if any, of the measures, and how the measures  could be supplemented or improved, we interviewed and reviewed related  documentation from the stakeholders previously mentioned to obtain their  opinions and analysis on the strengths and limitations of the measures and  any potential options suggested. We identified potential stakeholders  based on their expertise with broadband measures as evidenced by our  literature review, previous GAO work, recommendations from our internal  telecommunication experts, and suggestions by stakeholders we  interviewed to develop knowledge of issues related to broadband  performance measures. We also asked the stakeholders to discuss the  validity and reliability of the measures and any potential improvements.
We conducted this performance audit from February 2009 through  October 2009 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives. See appendix I for more  information about our scope and methodology.
subsections: section_title: Background
paragraphs: The Internet is a vast network of interconnected networks that is used by  governments, businesses, research institutions, and individuals around the  world to communicate, engage in commerce, perform research, educate,  and entertain. The Internet became widely accessible to U.S. households  by the mid-1990s. Early on, the primary means to access the Internet was a  dial-up connection, in which a standard telephone line is used to make an  Internet connection. A dial-up connection offers data transmission speeds  of up to 56 kilobits, or 1,000 bits per second (Kbps). Broadband access to  the Internet became available by the late 1990s. Broadband differs from a  dial-up connection in certain important ways. First, broadband  connections offer a higher-speed Internet connection than dial-up. For  example, some broadband connections offer speeds exceeding 1 million  bits per second (Mbps) both upstream (data transferred from the  consumer to the Internet service provider) and downstream (data  transferred from the Internet service provider to the consumer). These  higher speeds enable consumers to receive information much faster and  thus enable certain applications to be used and content to be accessed  that might not be possible with a dial-up connection. The higher  transmission speeds that broadband offers cost more than dial-up, and  some broadband users pay a premium to obtain very-high-speed service.  Second, broadband provides an “always on” connection to the Internet, so  users do not need to establish a connection to the Internet service  provider each time they want to go online. Although broadband often is  referred to as a singular service, it is available in a wide variety of data  speeds—ranging from 768 Kbps to greater than 100 Mbps. FCC’s current  categories for collecting data on the number of broadband subscribers by  advertised download and upload speeds range from greater than 200 Kbps  but less than 768 Kbps to equal to or greater than 100 Mbps. On August 20,  2009, as part of the proceeding to develop a National Broadband Plan, FCC  posted a public request for comment on defining “broadband.”
Consumers can receive a broadband connection to the Internet through a  variety of technologies that offer varying speeds of service, including, but  not limited to, the following:    Cable modem. Cable television companies first began providing  broadband service in the late 1990s over their cable networks. When  provided by a cable company, broadband service is referred to as cable  modem service. Cable modem service is primarily available in residential  areas. Cable modem service enables cable operators to deliver broadband  service by using the same coaxial cables that deliver pictures and sound to  television sets. Most cable modems are external devices that have two  connections, one to the cable wall outlet and the other to a computer or  router. Although the speed of service varies with many factors, download  speeds of up to 6 Mbps are typical. Cable providers are developing even  higher-speed services.
DSL. Local telephone companies provide digital subscriber line (DSL)  service, another form of broadband service, over their telephone networks  on capacity unused by traditional voice service. To provide DSL service,  telephone companies must install equipment in their facilities and install  or provide DSL modems and other equipment at customers’ premises and  remove devices on phone lines that may cause interference. Most  residential customers receive older, asymmetric DSL (ADSL) service with  download speeds of 1.5 Mbps to 3 Mbps. ADSL technology can achieve  speeds of up to 8 Mbps over short distances. Newer DSL technologies can  support services with much higher download speeds.
Fiber. This technology, also known as fiber optic, converts electrical  signals carrying data to light and sends the light through transparent glass  fibers smaller than the diameter of a human hair. Fiber optic systems can  transmit data at speeds far exceeding current DSL or cable modem speeds,  typically by tens of gigabits per second. Fiber optic technology may be  provided in several ways, including fiber to a customer’s home or business  or to a location somewhere between the provider’s facilities and the  customer. In the latter case, the last part of the connection to the  customer’s premises may be provided over cable, copper loop, or radio  technology. Such hybrid arrangements may be less costly than providing  fiber all the way to the customer’s premises, but they generally cannot  achieve the high transmission speed of a full fiber-to-the-premises  connection.
Satellite. Three providers currently offer broadband service via satellite in  the United States. These providers use geostationary satellites that orbit in  a fixed position above the equator and wirelessly transmit and receive data  directly to and from subscribers. Satellite companies provide  transmission from the Internet to the user’s computer and from the user’s  computer to the Internet, eliminating the need for a telephone or cable  connection. Typically a consumer can expect to receive (download) at a  speed of about 1 Mbps and send (upload) at a speed of about 200 Kbps.  Transmission of data via satellite causes a slight lag in transmission,  typically one-quarter to three-fourths of a second, thus rendering this  service less suitable for certain Internet applications, such as  videoconferencing. While satellite broadcast service may be available  throughout the country, it generally costs more than most other  broadband modes and its use requires a clear line of sight between the  customer’s antenna and the southern sky. Both the equipment necessary  for service and recurring monthly fees are generally higher for satellite  broadband service, compared with most other broadband transmission  modes.
Wireless. Land-based, or terrestrial, wireless broadband connects a home  or business to the Internet using a radio link. Some wireless services are  provided over unlicensed radio spectrum and others over spectrum that  has been licensed to particular companies. In licensed bands, some  companies are offering fixed wireless broadband throughout cities. Also,  mobile telephone carriers—such as the large companies that provide  traditional cell phone service—have begun offering broadband mobile  wireless Internet service over licensed spectrum—a service that allows  subscribers to access the Internet with their mobile phones or laptops in  areas throughout cities where their provider supports the service. A  variety of broadband access technologies and services also are provided  on unlicensed spectrum—that is, spectrum that is not specifically under  license for a particular provider’s network. For example, wireless Internet  service providers may offer broadband access in particular areas by  establishing a network of subscriber stations, each with its own antenna  that relays signals throughout a neighborhood and has a common interface  to the Internet. Subscribers place necessary reception equipment outside  their homes that transmits and receives signals from the nearest antenna.  Also, wireless fidelity (Wi-Fi) networks—which provide broadband service  in so-called hot spots, or areas within a radius of up to 300 feet—can be  found in cafes, hotels, airports, and offices. Such netowrks generally use a  short-range technology that provides speeds up to 54 Mbps. Some  technologies, such as Worldwide Interoperability for Microwave Access  (known as WiMAX), can operate on either licensed or unlicensed bands,  and can provide broadband service up to approximately 30 miles.
FCC has primary responsibility for regulating broadband. Section 706 of  the Telecommunications Act of 1996 directs FCC to encourage the  deployment of advanced telecommunications capability, which includes  broadband, to all Americans. Under this authority, FCC has to date  established a minimal regulatory environment for broadband   Internet access services. In the past, FCC has stated that less   regulation has encouraged providers to invest in broadband  infrastructure. The Communications Act, as amended, allows FCC to  classify services as telecommunications services or information  services, the latter being subject to fewer regulatory restrictions. FCC,  through a number of proceedings, classified broadband Internet access  (regardless of the platform) as an information service.  FCC does not  have explicit statutory authority to regulate the provision of information  services; however, FCC has the authority to impose regulations under  what is termed its ancillary jurisdiction to regulate services that are  reasonably related to its existing statutory authority. FCC also has the  authority to adopt broadband regulations to ensure that broadband  providers are capable of providing authorized surveillance to law  enforcement agencies.
As part of its responsibilities, FCC has periodically issued a report to  Congress on the status of advanced telecommunications capability in the  United States, including the quality of broadband data. To assist in the  preparation of this report, in 2000, FCC implemented the previously  described broadband reporting form, a semiannual reporting requirement  for facilities-based broadband Internet service providers. In November  2004, FCC modified its rules on filing this information, and the revised  rules went into effect for the companies’ second filing in 2005. Specifically,  FCC removed existing reporting thresholds, and all companies were  required to report their total state subscribership by technology. In 2006,  we reported that the approach FCC then used to collect data on  broadband deployment, which counted broadband service providers with  subscribers at the ZIP code level, resulted in inadequate information about  broadband deployment. Subsequent to our recommendation, in March  2008, FCC acted to increase the precision and quality of its broadband data  by revising its methodology and requiring that broadband providers report  the number of broadband connections in service by census tract.
In addition to FCC’s data collection effort using its broadband reporting  form, the Broadband Data Improvement Act calls for additional actions to  improve the quality of data available on broadband deployment. Among  other things, the act directs FCC to  (1) periodically survey consumers to collect information on the types  of technologies used by consumers to access the Internet, the  applications or devices used in conjunction with broadband  service, and the actual connection speeds of users;  (2) collect information on reasons why consumers have not  subscribed to broadband services;  (3) determine certain demographic data for geographical areas not  served by any provider of advanced telecommunications capability  (i.e., areas where broadband has not yet been deployed); and  (4) provide information on the extent of broadband service capability,  including the speed and price of broadband service in a total of 75  communities in at least 25 countries.
FTC also has regulatory jurisdiction over broadband services with respect  to competition and consumer protection issues. FTC’s jurisdiction over  broadband services comes chiefly from its statutory mandate to prevent  “unfair methods of competition” and “unfair or deceptive acts or practices  in or affecting commerce” under FTC’s enabling legislation, the FTC Act.  Although this authority is very broad, certain limited market sectors are  expressly excluded from FTC’s enforcement authority. In particular, FTC’s  enforcement authority does not reach “common carriers subject to the  Communications Act of 1934, as amended. However, since most  broadband Internet services are not provided on a common carrier basis,  they are generally part of the larger economy subject to FTC’s general  competition and consumer protection authority with regard to methods,  acts, or practices in or affecting commerce. FTC has, where appropriate,  investigated and brought enforcement actions in matters involving access  to content via broadband and other Internet access services.  Additionally, FTC has brought a variety of cases against Internet service  providers that have engaged in allegedly deceptive marketing and billing  practices.
Two other federal agencies have responsibility for telecommunications  policies. The Office of Science and Technology Policy (OSTP) within the  Executive Office of the President has a broad mandate to advise the  President and the federal government on the effects of science and  technology on domestic and international affairs and has led interagency  efforts to develop science and technology policies and budgets. NTIA is  the President’s principal telecommunications and information adviser and  works with other executive branch agencies to develop the  Administration’s telecommunications policies.
section_title: Multiple Measures Are Available for Consumers, Industry, Government, and Others to Assess Broadband Performance
subsections: section_title: Consumers Can Generally Access Measures of Availability, Price, Advertised Speed, and Actual Delivered Speed from Broadband Providers and Third Parties
paragraphs: Although there are limitations that we discuss later, consumers interested  in broadband service can generally contact providers or search provider  Web sites to determine the availability of service, advertised price, and  advertised speed of broadband service in their area. For example,  consumers can go to att.com or timewarnercable.com and enter their  street address to learn about the availability of broadband service at their  address, including price and advertised speeds. Each Web site also  provides a phone number that consumers can use to reach a customer  service representative to obtain information on availability, price, and  advertised speeds of service. Consumers can then make their own  comparisons of these prices and advertised speeds. In addition, third  parties provide consumer Web sites, such as dslreports.com, that assemble  this information for consumers to review.
However, actual delivered speeds depend on multiple factors, such as the  equipment of the consumer, the applications in use, and Internet traffic,  and may not always match advertised speeds or the theoretical maximum  speeds stated by the provider. Consequently, there are tools available to  consumers to measure actual delivered speed. Consumers with broadband  service have access to their actual delivered speeds through speed tests  from broadband provider Web sites and third parties. Speed tests generally  measure the “last mile” speed (download and upload) of the consumer’s  connection. Some third-party Web sites also provide information on actual  delivered speeds of service and allow consumers to compare speeds. For  example, speedtest.net allows individuals to compare their speed with that  of other consumers by provider or in a set geographic region.
Some states have also completed broadband mapping efforts that provide  consumers with information on broadband performance, including  availability and advertised speed. We previously reported that 12 states  had mapped broadband deployment, and 2 of these states, California and  Massachusetts, had mapped both the speed and availability of broadband  in their state and placed the information on their state’s Web site. In its  2008 report, California also provided information on average delivered  upload and download speeds aggregated throughout the state and  advertised residential speeds by price.
section_title: Industry and Government Have Some Broadband Performance Measures to Make Comparisons across Various Segments of the United States
paragraphs: The stakeholders we interviewed told us that FCC’s broadband data,  collected through its broadband reporting form, constitute the primary  data source generally used to measure performance and make  comparisons across various segments of the United States, although there  are limitations, which we discuss later. The Commission has tracked  broadband subscribership and deployment since 2000 through its  broadband reporting form. In 2006, we reported that the approach FCC  then used to collect data on broadband deployment, which counted  broadband service providers with subscribers at the ZIP code level,  resulted in inadequate information about broadband deployment. To  improve this information, in 2008, the Commission revised the semiannual  reporting requirements of the broadband reporting form. The Commission  now requires most broadband providers to file subscribership information  by census tract, including the number of subscribers by technology, speed  tier, and business/residential connection. In addition, mobile wireless  service providers are now required to report the number of connections in  (1) individual states, (2) the census tracts that best represent their  broadband service footprint, and (3) in a separate category, the number of  subscribers whose device and subscription permit them access to the  lawful Internet content of their choice. These changes are expected to  result in data that are more detailed than what was previously collected.  The first round of data filings under the new requirements was due on  March 16, 2009. As of September 2009, FCC staff was still in the process of  analyzing the information.
Stakeholders also identified the Pew Internet & American Life Project’s  reports on home broadband adoption as a source for measuring adoption  and making comparisons across the United States. The results in the  reports are based on data from approximately 2,300 telephone interviews  (on both landline and cellular telephones) conducted by Princeton Survey  Research International over the course of a month. The 2009 report  included the following information that can be used to compare rural and  nonrural areas: broadband adoption, broadband connection type, and,  when applicable, reason for not having broadband access or Internet  access.
section_title: Industry, Government, and Other Stakeholders Have Many Performance Measures Available for Making International Comparisons
paragraphs: Through our literature review and interviews with stakeholders, we  focused on 10 performance measures often used by industry, government,  and other stakeholders to make international comparisons of broadband  service, as summarized below (limitations of these measures are discussed  later). These measures fall into two general categories: (1) broadband- specific measures and (2) more general measures that cover a wide array  of information and communications technology (ICT).
The broadband-specific rankings measure a nation’s broadband  performance by focusing on the availability, penetration (or adoption), and  quality of broadband in each country, and include those listed below (see  table 1 for the U.S. ranking for each.)
Broadband Adoption Index. The Phoenix Center for Advanced Legal and  Economic Public Policy Studies recently developed the Broadband  Adoption Index (BAI), which proposes to compare the actual value that a  society derives from broadband usage with that country’s target level for  adopting various broadband technologies based on maximizing societal  well-being. These targets vary by technology, demographic group, and  country. The index does not include an overall ranking of countries based  on broadband performance, because each country has its own unique set  of adoption targets.
Broadband Quality Score. The Oxford Saïd Business School in Oxford,  United Kingdom (UK), in conjunction with the University of Oviedo in  Oviedo, Spain, and Cisco Systems, Inc., created the Broadband Quality  Score (BQS) in September 2008 to highlight each representative nation’s  ability to benefit from next-generation Web applications and services.  According to the study, to establish broadband leadership, countries must  focus on broadband availability, penetration, and quality.
Broadband Subscribers per 100 Inhabitants. OECD produces many  broadband-related measures annually on its online broadband Web site.  According to FCC and many of the stakeholders we interviewed, one of  the most widely reported figures on broadband performance is OECD’s  count of broadband subscribers per 100 inhabitants by technology. OECD  also collects comparative data from its 30 member countries on multiple  broadband measures such as penetration, usage, coverage, prices, services  and speeds, and choice and competition. However, unlike other  stakeholders, OECD does not aggregate its data into a composite indicator  of national broadband performance.
Broadband Performance Index. The European Commission recently  implemented the Broadband Performance Index (BPI), which measures  and benchmarks the overall broadband performance of European Union  member states based on a range of factors, which could include speeds,  rural coverage, affordability, innovation, and other socioeconomic  dimensions. In particular, the BPI ranks the EU-27 countries plus Norway  in terms of supply and demand factors that affect the penetration and use  of broadband.
ITIF broadband rankings. For its broadband rankings, the Information  Technology and Innovation Foundation (ITIF) measures three primary  broadband indicators, household penetration (rather than subscriber),  average speed, and price, to rank the broadband performance of OECD  nations. ITIF notes the importance of non-policy factors on a nation’s  broadband performance, including demographic, economic, and  broadband supply variables.
In contrast to broadband-specific rankings, the other performance  measures we identified were based on each country’s development and  use of ICT. These rankings are more general, focusing on the larger picture  of how ICT usage, infrastructure, and skills can affect a country’s  economic growth. According to an official at the Technology and Policy  Institute (TPI), broadband is but one component in the makeup of a  country’s ICT landscape, as ICT encompasses Internet usage along with  other forms of telecommunications. According to FCC, these various  measures demonstrate the value of understanding the broader context  when making comparisons regarding broadband deployment and  adoption. Examples of ICT-specific rankings include the following:    Connectivity Scorecard. The Dean at the Haskayne School of Business at  the University of Calgary in Calgary, Canada, worked in collaboration with  Nokia Siemens Networks and LECG (a global services and consulting  firm) to release the first version of the Connectivity Scorecard in 2008. The  scorecard measures the impact of ICT on economic growth in three key  areas of society—the consumer sector, the business sector, and the  government sector. The report presents separate sets of rankings for  “innovation-driven economies” and “resource- and efficiency-driven  economies” while specifically focusing on each country’s ICT  infrastructure and usage.
E-readiness Ranking. The Economist Intelligence Unit (EIU) is the  business information arm of The Economist Group, publisher of The  Economist magazine. The EIU produces an annual E-readiness Ranking,  which measures the quality of a country’s ICT infrastructure as well as the  ability of its consumers, businesses, and government to use ICT to their  benefit. The EIU makes this assessment by specifically measuring a  country’s connectivity and technology infrastructure, business  environment, social and cultural environment, legal environment,  government policy and vision, and consumer and business adoption.  Overall, more than 100 separate qualitative and quantitative criteria are  considered.
Networked Readiness Index: The World Economic Forum, in  cooperation with INSEAD international business school’s eLab research  center and Cisco Systems, Inc., produced the Networked Readiness  Index (NRI). The NRI is used to assess the extent to which different  economies benefit from the latest ICT advances based on their ICT  environment, readiness, and usage while taking into account the key roles  played by individuals, businesses, and governments. The NRI covers 134  economies worldwide and accounts for nearly 70 factors.
International Communications Market. In its 2008 report, Ofcom, the  regulator for the UK communications industry, described developments in  international communications markets, including information on  broadband availability and usage. In the report, Ofcom aimed to provide  statistically driven international comparative data for the UK  communications sector by examining trend data from 2002 to 2007 on how  various countries’ industries, consumers, and regulatory landscapes affect  their communication markets.
ICT Development Index. The International Telecommunication Union  (ITU), a United Nations agency, developed the ICT Development Index  (IDI), which measures the development of ICT, the level of advancement  of ICT, and the development potential of ICT in more than 150 countries  worldwide, comparing their progress between 2002 and 2007. The purpose  of the index is to track the global digital divide and to measure each  country’s progress toward becoming an “information society.” The  primary index measures ICT infrastructure/access, use, and skills, while a  separate index was created to capture the price of ICT relative to a  country’s income.
Eight of the 10 performance measures listed above are “composite  indexes,” i.e., combinations of measures that are generally used to try to  account for and normalize a variety of factors such as demographic,  economic, and geographic differences among countries, which according  to many of the stakeholders we spoke with can affect broadband  deployment and penetration. Several of the stakeholders identified  advantages to using composite indexes in making international  comparisons. Officials from the European Commission reported that  composite indexes are a useful tool to summarize the multidimensional  issues, such as the socioeconomic differences among countries, which  cannot be captured by a single indicator. According to ITU, compared with  single indicators, composite indexes allow grouping several key  performance indicators into one figure that captures a variety of  information and provides a more comprehensive picture. While the  various indexes differ on which demographic, economic, and geographic  factors play a greater role in the supply and demand of broadband,  income, age, education, population density, gross domestic product (GDP)  per capita, and intermodal competition are generally considered  important. According to ITIF, nonpolicy factors, such as demographic,  economic, and broadband supply variables explain about three-quarters of  the differences among nations’ broadband performance in international  rankings. The determination of which factors to include or exclude in a  composite index can greatly affect a nation’s ranking in a report, as  demonstrated by the fact that the United States’ broadband and ICT  rankings vary greatly by study, as shown in table 1.
section_title: Stakeholders Reported Limitations with Current Broadband Performance Measures, but Views Are Mixed on Potential Alternatives and Ongoing Efforts Need Improvement
subsections: section_title: Price, Actual Delivered Speed, and Service Reliability Measures for Consumers Have Limitations, but There Are Differing Views on Alternatives
paragraphs: Even though consumers have access to measures, stakeholders told us  that measures of price, actual delivered speed, and service reliability have  limitations that may affect their usefulness for consumers:    Price. Stakeholders told us the available pricing measures for consumers  are limited. For example, officials from the Consumer Federation of  America and Pew Internet & American Life Project told us the lack of a  comprehensive and consistent measure from the government for  consumers to compare prices from providers was a limitation. They added  that improved measures for prices would help consumers make more  informed decisions about broadband services. Although FCC has open  proceedings on requiring providers to include measures of price in the  broadband reporting form, it currently does not collect this information.
Actual delivered speed. Stakeholders also identified limitations regarding  the speed tests for consumers to measure actual delivered speeds. A  representative from Akamai, a company that handles approximately 15 to  20 percent of all Internet traffic worldwide through its global server  network, said one problem with speed tests is that the result can be  significantly affected by the location of the server that is used to test the  speed; the farther away the server, the less accurate the result. Many other  factors can also affect a user’s speed of service, such as congestion on the  network, time of day, and other applications that the user may have open  on the computer when testing. NTIA officials told us that the speed tests  are not able to determine the Internet traffic congestion points, if any,  along the chain of networks. An official from the Pew Internet & American  Life Project told us the results of the speed tests are not verified by other  parties. He also explained that some third-party Web sites that attempt to  compare actual delivered speeds have limited numbers of respondents and  do not have an independent party verify the results, a fact that decreases  the utility of the information for making comparisons. Finally, an official  from the Information Technology and Innovation Foundation said the lack  of comprehensive data for consumers to compare actual delivered speeds  from providers was a limitation for consumers in comparing service  options and policy makers in monitoring broadband. Actual delivered  speed can be an important measure for consumers because it can  determine whether or not a connection can be used to originate and  receive high-quality voice, data, graphics, and video. FCC has open  proceedings on requiring providers to report actual delivered speeds on  the broadband reporting form, but it currently does not collect this  information.
While broadband connection speeds that customers experience are  generally not identical to the advertised speeds or theoretical maximums  offered by the broadband provider, there is some evidence that consumers  are not focused on this issue. Despite access to the tools to measure actual  speed, one study found that few people actually know the speed of their  broadband connections. In its report titled “Home Broadband Adoption  2006,” the Pew Internet & American Life Project reported that 81 percent  of broadband users did not know their home connection speed. In  addition, the federal government has received relatively few complaints  regarding broadband speed. From February 1, 2008, through May 12, 2009,  FCC reported receiving about 624,000 informal complaints, of which only  157 were related to broadband speed. Further, FTC reported receiving  approximately 147 complaints that could be related to broadband speeds  from January 2005 through June 19, 2009. According to some  stakeholders, such as the Information Technology and Innovation  Foundation, consumers appear more concerned with their end user  experience, such as the ability to complete transactions or use their  applications.
Service reliability. Some stakeholders we contacted, including  BroadbandCensus.com, IEEE (previously known as the Institute of  Electrical and Electronics Engineers), the Internet Engineering Task  Force, Akamai, an economist from the Massachusetts Institute of  Technology (MIT), NTIA, and Wireless Internet Service Provider  Association (WISPA) are concerned that there is no measure for  consumers that addresses service reliability. A service reliability measure  would provide information to consumers on factors such as transmission  quality, which affects perceived speed and could be useful to consumers in  comparing the reliability of broadband services. According to an official  from Akamai, service quality is the most difficult performance measure to  define, measure, and relay to a consumer.
While consumers have measures of price, advertised speed, and actual  delivered speed to make decisions regarding broadband service, some  stakeholders suggested improved measures of price and actual delivered  speeds for consumers, as shown in table 2.
As shown in table 3, stakeholders identified arguments for and against the  proposed measures.
It should be noted that while federal and state agencies and public/private  partnerships, academicians and think tanks, consumer advocacy groups,  and trade and industry groups identified arguments for and against the  proposed measures, broadband providers generally only provided  arguments against the proposed measures. Thus, while stakeholders  identified multiple alternatives, they differed on the need for FCC to  develop additional reporting requirements to measure price, average  actual delivered speed, and service reliability as follows:    Consumer advocacy groups and academicians and representatives from  think tanks generally believed there was a need for improved information  on price and actual delivered speeds to make comparisons and good  decisions about service. These stakeholders preferred that FCC require  broadband providers to report price per megabit per second and the  averaged actual delivered speed of last-mile connections (from the home  to the first provider node or aggregation point) to provide more consistent  measures for consumers to make comparisons. These stakeholders  generally believed that calculating price per megabit should be done using  the published, stand-alone nonpromotional, noncontractual price. Some  suggested providing an average price by speed tier, while others suggested  providing the lowest and highest prices by speed tier. Finally, some  consumer advocacy groups and academicians and representatives from  think tanks also favored a measure on service reliability to provide  consumers with information on the quality of their connections.
In contrast, broadband providers and trade and industry groups generally  did not perceive a need for additional broadband measures because, in  their opinion, price and speed information is readily available from  providers and third-party sources. According to these stakeholders,  additional reporting requirements would be an intrusion into a market that  is working, as evidenced by falling prices for increased speeds. They  added that additional reporting requirements would be an impediment to  investment in infrastructure, as more resources would need to be devoted  to data collection. These stakeholders also reported that price per megabit  and the average actual delivered speed are difficult to measure (as  previously shown in table 3), and that FCC is not likely to report the  information in a timely fashion. For example, in the past, it has taken FCC  close to a year to report the data from the broadband reporting form once  it has been submitted by broadband providers.
While officials at federal and state agencies and public-private  partnerships generally said more information is good, there were mixed  opinions on the need for FCC to require additional broadband measures.  None of the federal agencies we interviewed provided an opinion; an  official with the California Public Utilities Commission was uncertain if  additional requirements were needed because similar information is  already available to the public; and of the two interviewed, one  public/private partnership was for additional broadband measures and one  was against.
Finally, all stakeholder groups generally noted that FCC’s efforts to  develop periodic surveys, per the Broadband Data Improvement Act, and  a voluntary registry for consumers to report information about their  broadband service, could be used to collect and disseminate price and  speed information for consumer use. However, stakeholders also  cautioned that periodic consumer surveys and a voluntary registry may  not provide reliable information because consumers are not informed  enough about the price and speed of their broadband service to report  accurate information, and they believe that this should be taken into  consideration when reviewing the results. Additionally, consumers may  not take the time to enter their information in a registry, as current  voluntary registries for broadband data sponsored by third parties are  sparsely populated.
section_title: Current Data Sources for Industry and Government to Compare Broadband across Various Segments of the United States Have Limitations, and Stakeholders Generally Support Ongoing Efforts for Improvement
paragraphs: Despite FCC’s efforts to improve the data collected through its broadband  reporting form, comparisons of broadband service across various  segments of the country still have the following limitations that diminish  their usefulness in informing policy and investment decisions:    While FCC requires most broadband providers to report broadband  subscribership on the broadband reporting form, it does not have a  reporting requirement for these providers to report broadband  availability. Additionally, although the majority of those we interviewed  cited the change from reporting by ZIP codes to census tract as an  improvement, some said the data still do not provide enough granularity to  track subscribership in tribal lands or rural areas. In fact, according to  FCC’s report Bringing Broadband to Rural America: Report on a Rural  Broadband Strategy, there are no accurate data on broadband deployment  in rural America, including where broadband facilities are deployed,  prices, speeds, and the number of subscribers.
FCC also does not require broadband providers to report price  information for broadband services on its broadband reporting form, so it  is difficult to measure how price varies across various segments of the  country. The Commission has open proceedings concerning whether and  how the Commission could collect price information for broadband  services. For example, the Commission sought comment on requiring  providers to report, for each state or each census tract in which they offer  service, the monthly price the provider charges for stand-alone broadband  service in each of the speed tiers used for the broadband reporting form,  potential alternatives, and whether and in what form the Commission  should use the reported service price information.
Similarly, FCC does not require broadband providers to include  information on actual broadband connection speeds experienced by  consumers, although the data from the revised broadband reporting form  will provide information on the number of connections by advertised  speed. As previously mentioned, actual delivered speed can determine the  applications that can be run by consumers and could be useful in  comparing broadband service across various segments of the country. The  Commission also has open proceedings concerning how the Commission  might require broadband service providers to report actual broadband  connection speeds, and any alternative means, in addition to or other than  requiring such service provider reporting.
Some stakeholders noted that FCC may overestimate the number of  wireless broadband users. FCC’s reporting requirement for mobile  wireless broadband service providers collects data on the number of  terrestrial mobile wireless subscribers whose subscription and device  allow them to access the Internet content of their choice, not the number  of consumers actually using broadband on the device. According to a Vice  President and Senior Fellow at the Technology Policy Institute, it is  unlikely that all persons whose subscription and device allow them to  access the Internet actually use the service. As a result, counts of the  number of terrestrial mobile wireless subscribers whose subscription and  device allow them to access the Internet content of their choice may  overestimate the number of wireless broadband users. However, other  stakeholders, such as an official with the Rural Utilities Services, thought  the reporting standard would produce accurate results, as they thought  most consumers that paid for the service would use it. Stakeholders we  spoke with generally characterized mobile wireless as a complement to  and not a substitute for fixed wireline service. They added that this may  change as the technology improves over time. Stakeholders also generally  agreed that the mobile wireless counts should be kept separate from fixed  wireline counts when determining deployment and availability.
Stakeholders also identified limitations with the Pew Internet & American  Life Project data. While the survey collects information on cost, speed,  availability, and usage, the data are limited because the sample size lacks  the granularity needed for making comparisons at the state or regional  level.
Despite the concerns about FCC’s data collected through the broadband  reporting form, several stakeholders said they found the data useful.  According to one academic expert, FCC’s broadband data are the best  publicly available data on the geographic dispersion of broadband services  across the United States. In addition, an official with a consumer advocacy  organization said FCC’s changes to the broadband data collection struck  the right balance between the need for detailed subscribership data and  the burden to providers of gathering such information by choosing the  census tract as the geographic unit for data collection.
To address the limitations in broadband data, recently enacted legislation  requires the Secretary of Commerce to obtain more complete data on  broadband availability. The Broadband Data Improvement Act requires the  Secretary of Commerce to establish a grant program for multiple  purposes, including collection of state-level broadband data. The  American Recovery and Reinvestment Act of 2009 requires NTIA to  establish a comprehensive nationwide inventory map of existing  broadband service capability and availability in the United States that  depicts the geographic extent to which broadband service is deployed and  available from a commercial provider or public provider throughout each  state. By February 17, 2011, NTIA must make the national inventory map  available online to the public in a form that is interactive and searchable.  The Recovery Act provides up to $350 million, pursuant to the Broadband  Data Improvement Act, for developing and maintaining the national  broadband inventory map.
NTIA has used the grant-making authority provided under the Broadband  Data Improvement Act to establish the State Broadband Data and  Development Grant Program. Through this program, NTIA has solicited  grant applications from states for projects designed to collect data,  develop state maps, conduct state planning efforts, and deliver data to  NTIA for the purposes of developing the national broadband map. As of  September 9, 2009, NTIA had received applications representing all 50  states, 5 territories, and the District of Columbia. NTIA is currently  reviewing the applications and plans to announce funding decisions  beginning in early fall 2009. Applicants must demonstrate that they have  the ability to provide a substantially complete set of all broadband  mapping data on or before February 1, 2010, and to complete such data  collection by March 1, 2010. NTIA officials told us they are working  closely with FCC regarding the development of the map. As part of its  efforts, NTIA is requiring awardees under the State Broadband Data and  Development Grant Program to provide, among other things, the following  information:  for each facilities-based provider of broadband service, a list of all census  blocks of 2 square miles or smaller in which broadband service is available  in the provider’s service area;  for census blocks of greater than 2 square miles, for each facilities-based  provider of broadband service, a list of all street segments in the census  block in which broadband service is available in such provider’s service  area;  for wireless providers, geographical information system compatible  polygonal shape files depicting areas in which broadband service is  available;  technology type of service provided by census block, street segment, or  shape file area, as applicable;    maximum advertised speed available across each service area or local  franchise area, by metropolitan or rural statistical area;    actual delivered speed that can be consistently achieved during expected  periods of heavy network usage by census block, or street segment, as  applicable; and    middle-mile connection points.
Though the program does not require it, awardees may satisfy program  requirements by providing address-level data. Awardees may also provide  last-mile connection points, if available. Identification of a provider’s name  and its availability/speed at a particular address is considered confidential.  However, identification of a service provider’s specific service area, or  “footprint,” at the census block or street segment level is not considered  confidential and will be displayed on the national broadband map. The  initial period of performance for awards under the program was 5 years  from the date of the award. However, on September 10, 2009, NTIA  announced that it will fund the mapping and data collection efforts for 2  years from the date of the award and will assess lessons learned,  determine best practices, and investigate opportunities for improved data  collection prior to obligating funding for subsequent years.
In the notice of funds availability for the State Broadband Data and  Development Grant Program, NTIA noted that it reserved the right to  request that FCC exercise its authority to compel any service provider  subject to its jurisdiction to provide data. NTIA also explained that, to the  extent possible, the service areas of individual providers will be  aggregated with those of other providers of the same technology type.  According to NTIA officials, this determination was based on its review of  the comments, an examination of mapping methodologies employed at the  state level, and consultation with FCC.
Stakeholders generally agreed that the national broadband inventory map  would help supplement gaps in FCC’s broadband data by providing  detailed data on availability and subscribership across the country. For  example, a Pew Internet & American Life Project official told us that  broadband mapping has the most potential for providing the granular and  accurate information required to make comparisons across the country.  Several stakeholders also explained that in order for the national  broadband map to be effective, NTIA needs to develop data collection  standards to help ensure that the data collected by each state are  comparable across states. Some stakeholders also stressed the need for  collecting information regarding demand side data (desire for service or  usage).
Despite the consensus among stakeholders regarding the potential  benefits of broadband mapping, there are some concerns about the effort.
We found NTIA did not provide guidance on how to calculate actual  delivered speed that can be consistently achieved during expected periods  of heavy network usage at the address. For example there is no guidance  on the number of speed measurements that must be taken or a definition  of heavy network usage. We have previously reported that consistency— the extent to which data are collected using the same procedures—is a key  dimension of data quality and a key attribute of a successful performance  measure. NTIA officials told us they chose not to provide this guidance  because each provider may have a different method for measuring speed,  and they did not want to prescribe a standard method, given the multiple  technologies used. However, this could result in inconsistent  measurements across grantees, limiting the effectiveness of the mapping  effort in making comparisons across the country.
While NTIA required applicants to provide a description of the methods  the applicant intends to employ to verify data accuracy, it did not set out  specific standards on how to do so. NTIA’s notice of funds availability did  provide the following example: “A project should propose to collect  availability data by address . . . and should cross-check that data for  accuracy by using at least one other metric.” We have previously  reported that both verification and validation of performance data are  important for reducing the risk of producing inaccurate data; this  additional information helps to place the credibility of an agency’s  reported performance data in context for decision makers. NTIA offic told us they chose not to specify how grantees should verify data becauser  they did not want to be too prescriptive, as allowing states to develop thei own data verification processes may yield best practices that can be used  going forward. While it is too early to determine the effect, if any, of the  limited guidance, the lack of specific standards for data verification could  result in inconsistent data across states, limiting the effectiveness o data in making comparisons acros s the country.
The broadband providers we spoke with were generally concerned about  the cost and burden of complying with any additional reporting  requirements. For example, officials from Time Warner told us that some  providers do not store data in an address-by-address format and would  have to revise their existing data collection procedures, taking time and  resources away from network upgrades. According to FCC, broadband  providers already average 337 staff hours to complete the reporting  requirements for the broadband reporting form. Other stakeholders, such  as Connected Nation, Consumers Union, and the Organization for the  Promotion and Advancement of Small Telecommunications Companies  (OPASTCO), also acknowledged that additional reporting requirements  can be particularly burdensome to small broadband providers in rural  areas that do not have the staff and resources of larger broadband  providers. In addition, the NTIA requirement to provide data on  availability may overlap with FCC’s requirement for broadband providers  to report subscribership information through the broadband reporting  form, because subscribership is a subset of availability. Service must be  available for a consumer to be a subscriber. To ease the potential burden  on broadband providers, NTIA has timed its future data collection efforts  to coincide with FCC’s broadband data collection.
Finally, some stakeholders, including the Pew Internet & American Life  Project, Consumer Federation of America, and Consumers Union, were  concerned that some data underlying the state maps would not be publicly  available for review. They explained that public-private partnerships often  agree to nondisclosure agreements with broadband providers to facilitate  data collection by easing provider concerns regarding what the providers  consider to be the proprietary nature of the data. However, according to  these stakeholders, this reduces the transparency of the maps and  prevents other interested parties from analyzing the information.
Again, stakeholders generally noted that FCC’s efforts to develop periodic  surveys (per the Broadband Data Improvement Act) and a voluntary  registry could be used to collect and disseminate price and speed  information to make comparisons of broadband service across the  country. But they cautioned that information gleaned from these efforts is  limited and therefore should be a supplement to other data collection  efforts, because, as previously mentioned, consumers may not be well  informed about the price and speed of their Internet service.
section_title: International Broadband Comparisons Have Limitations for a Variety of Reasons, but Stakeholders Generally Support FCC’s Efforts to Develop Additional International Comparisons
paragraphs: As previously discussed, stakeholders reported that socioeconomic  differences among countries can limit the efficacy of international  comparisons. For example, OECD and ITU report broadband subscribers  per 100 inhabitants rather than as a percentage of households. According  to a senior official at the Technology Policy Institute, household size alone  explains most of the differences in the broadband rankings of countries,  since countries with larger households are likely to have lower per capita  residential connections. As the Phoenix Center demonstrated, even if  every home and business in every OECD country were wired with a  broadband connection, the United States’ per capita rank would fall from  15th to 20th because the United States has a larger average household size  than countries, such as Sweden and Iceland, that rank above it.  According to FTC staff, because the socioeconomic status of individual  countries and the historical nature of their Internet access markets can  vary widely, simple comparisons of individual indicators such as  broadband deployment and adoption rates across countries may not be  meaningful.
In contrast to OECD’s use of subscriber data, the composite indexes we  previously described attempt to take into account the socioeconomic  differences and other variables among countries when comparing  broadband performance. However, according to stakeholders, even  composite indexes provide limited analysis because of their complex  nature and the number of variables they seek to measure. For example,  one of the authors of the Connectivity Scorecard noted that composite  indexes are “ultimately based on subjective decisions about which  indicators to include or exclude and how to weight these indicators.” The  more factors or variables considered in a composite index, the more data  must be collected, normalized, and weighted for comparative purposes. A  spokesperson for the EIU’s E-readiness ranking stated that more variables  increase the room for error. Multiple variables also make it difficult to  determine a causal relationship for policy-making purposes between the  variable and its measured impact on the result, according to officials with  the European Commission. For example, the EIU included nearly 100  quantitative and qualitative variables in its E-readiness Ranking Report in  an attempt to measure the impact of a country’s social, political,  economic, and technological developments on its ICT usage and  infrastructure. A representative of the EIU told us that there are  limitations to this approach, and that some of the unit’s data must be  estimated because of the sheer number of variables the EIU attempts to  consider for the 70 countries in the E-readiness Ranking Report.
Stakeholders also reported that the necessary data to improve  international comparisons of broadband deployment and penetration are  not available. OECD and others have noted that while supply-side data  from broadband providers are both readily available and easily  quantifiable, demand-side data from consumers for measuring broadband  penetration are limited. Some stakeholders, such as officials with ITU, TPI,  and ITIF, have noted the importance of collecting demand-side data  through household surveys to more accurately reflect how consumers use  their personal broadband service for economic or social gain.  Governments are also increasingly recognizing the importance of  collecting better demand-side data. For example, EU member countries  are now required to collect household survey data on ICT usage.
In addition, stakeholders reported a lack of uniformity and reliability with  the data used to make international broadband comparisons, whether by  composite index or single indicator. For example, although most of the  countries that participate in international broadband ranking systems  recognize broadband to be Internet service above 256 Kbps, there is no  internationally agreed upon definition for broadband, which affects the  comparability of the data collected. OECD and ITU have recommended  uniform reporting standards among their member countries, but the  standards are neither enforceable nor applicable to countries outside their  membership. In addition, some of the organizations that develop  international comparisons rely on participating countries to provide the  needed data rather than independently gather the data directly from  providers or in the form of household surveys, a fact that leads some to  question its reliability. The officials we interviewed from the organizations  that develop international comparisons told us they have limited ability to  corroborate the data received from participating governments, outside of  questioning and confirming a figure when a number appears out of line  with trend data. Estimates are also made when the data are simply lacking  for a particular country.
Currently, discussions are also taking place on how to collect and  differentiate among wireline, wireless, and mobile wireless broadband  counts. According to OECD, wireless Internet connections at broadband  speeds are increasingly available and particularly important in  underserved areas around the world. Similarly Internet access via mobile  cellular networks has grown rapidly with the increasing availability of  third-generation (3G) networks and enabled devices that allow users to  access the Internet over mobile cellular networks using a laptop, cell  phone, or alternative mobile device. A representative from the Economist  Intelligence Unit stated that mobile wireless Internet access is particularly  important for individuals in developing countries, such as in Africa, where  mobile access may be their primary Internet source. However,  stakeholders noted that it is important to differentiate between 3G  subscribers whose plan may allow them to access the Internet on their  mobile device and those who actually take advantage of the service;  current data usually do not differentiate and are therefore potentially  misleading. OECD is in discussions with member countries to develop a  common methodology to improve the collection of mobile wireless data.
Despite the concerns raised about the limitations of the measures used for  international comparisons, several stakeholders found the comparisons  useful. As previously mentioned, OECD’s count of broadband subscribers  per 100 inhabitants by technology is one of the most reported figures.  Representatives from the Consumer Federation of America, Free Press,  and the Pew Internet & American Life Project said the OECD broadband  comparisons provide valuable information to policy makers. In its  guidance on developing composite indicators, OECD noted that composite  indexes used by other organizations in making international broadband  comparisons are recognized as a useful tool in policy analysis and public  communication. The indexes serve the important purpose of raising  awareness among policy makers and the public of areas that deserve  particular attention in future policy decisions.
FCC has noted that a more fully developed picture of broadband markets  would provide more accurate and useful international comparisons. The  recent Broadband Data Improvement Act mandated that FCC include in  future 706 reports information that compares the extent of broadband  service capability in a total of 75 communities in at least 25 countries  abroad for each of the data benchmarks for broadband service under  FCC’s current speed tiers. The Commission was directed to choose  international communities for the comparison that will offer a population  size, density, topography, and demographic profile that are comparable to  those of various communities within the United States. In May 2009, FCC  officials informed us that they had already assembled a cross-bureau team  of economists and attorneys to perform this international comparison.  FCC staff is currently in the process of identifying and reaching out to a  number of countries believed to have the relevant broadband data  necessary to make such comparisons. According to the officials, they have  sent letters to 37 countries to request data. They are working under the  assumption that the mandate will require them to communicate the results  of their comparisons in the next Section 706 report, which is to be  released in February of 2010. In addition, on March 31, 2009, FCC posted a  public request for comment on the international comparisons component  of the act. The majority of stakeholders we spoke with generally support  FCC’s efforts to develop an additional international comparison on  broadband performance. Although the term “community” was not defined  in the act and had yet to be defined by FCC, this level of analysis could be  more granular and therefore more comparable than what is generally  provided in current international comparison reports. Representatives  from organizations such as Connected Nation and Free Press support data  that are collected and analyzed at a more granular local level rather than at  a national level, because they believe that such data make the comparisons  more relevant.
section_title: Conclusions
paragraphs: A wide range of measures to assess broadband performance is generally  available to consumers, industry, and government. However, many  stakeholders told us that the measures used by consumers and those used  to make comparisons across the United States and among other countries  have limitations. Reaching a compromise among broadband providers,  consumer advocates, and others on improved broadband measures in the  United States has proven to be difficult because they do not agree on  alternatives for improvement. Nevertheless, all stakeholders are generally  supportive of NTIA’s State Broadband Data and Development Grant  Program and its effort to create a national broadband inventory map,  which could help fill some current gaps in data. NTIA has made progress  in (1) implementing its State Broadband Data and Development Grant  Program and (2) requiring grantees to collect data that have important  implications for consumers, policy makers, and industry in measuring  broadband performance. NTIA will begin receiving data by March 2010 as  part of its new grant initiative to collect state-level broadband data and  establish a national broadband inventory map. However, NTIA lacks  specific guidance for grantees on calculating actual delivered speeds.  Without such guidance, it will be difficult to ensure the consistency, and  therefore the quality, of the data, limiting the effectiveness of the mapping  effort in making comparisons across the country. In addition, while NTIA  provided potential grantees with an example of how to verify data  accuracy, it did not provide specific standards to verify data accuracy.  Consequently, NTIA will need to determine whether the data provided in  the initial submission are accurate, and if additional guidance is needed.  Developing procedures to help ensure consistent and accurate data is  critical, as NTIA begins to distribute funds to grantees and they begin their  data collection. More importantly, this effort has the potential to provide  consumers, policy makers, and industry with accurate and reliable  information such as broadband availability, type, and advertised and  actual delivered speed by census block, information that could be used by  each in their decision-making process and help guide broadband  investment in unserved or underserved populations.
section_title: Recommendation for Executive Action
paragraphs: To increase the data quality and subsequent results from the State  Broadband Data and Development Grant Program, including a searchable  nationwide inventory map of existing broadband service capability and  availability in the United States, we recommend the Secretary of  Commerce examine the first round of data collection and determine  whether to develop specific guidance for grantees to improve the  consistency and accuracy of the data collected under the program.
section_title: Agency Comments
paragraphs: We provided a draft of this report to the Department of Commerce and  FCC for their review and comment. The Department of Commerce  provided written comments, which are reprinted in appendix II. In its  written comments, the Department of Commerce generally agreed with  our recommendation and stated that it had already begun taking actions to  address the recommendation. More specifically, the Department of  Commerce stated that immediately following the awarding of grant funds,  it will investigate opportunities for improved data collection methods  including qualitative and quantitative analyses of data collection and  verification methods, as well as an assessment of which methods are cost- efficient and accurate. FCC responded that it did not have any comments  on the draft report.
We are sending copies of this report to the Secretary of Commerce and the  Chairman of the Federal Communications Commission. In addition, the  report will be available at no charge on the GAO Web site at  http://www.gao.gov.
If you or your staff have any questions about his report, please contact me  at (202) 512-2834 or goldsteinm@gao.gov. Contact points for our Offices of  Congressional Relations and Public Affairs may be found on the last page  of this report. Contact information and major contributors to this report  are listed on appendix III.
section_title: Appendix I: Scope and Methodology
paragraphs: To gather information related to both objectives, we reviewed related  documentation and laws, including the Broadband Data Improvement Act,  enacted in 2008; the legislative history of the act; the American Recovery  and Reinvestment Act of 2009 and its legislative history; various Federal  Communications Commission (FCC) proceedings; and reports from the  Congressional Research Service (CRS). We also conducted a literature  review to identify broadband performance measures, including  international broadband comparisons.
To identify the broadband performance measures available to consumers,  industry, government, and other stakeholders, we interviewed officials and  representatives from several stakeholder groups. On the basis of the  requirements of the mandate, the literature review, the judgment of our  staff with expertise in broadband and telecommunications issues, and  suggestions from the initial interviews held, we determined to include the  following stakeholder groups in our analysis to ensure a variety of  perspectives and views on broadband performance measures:  academicians and think tanks, broadband providers, consumer advocacy  groups, federal and state agencies and public/private partnerships,  international organizations, and trade and industry groups. We used the  same process to identify potential stakeholders for interviews. Table 4  contains a detailed list of the stakeholders included in our study:  To evaluate the limitations, if any, of the measures, and how the measures  could be supplemented or improved, we interviewed and reviewed related  documentation from the stakeholders previously mentioned to obtain their  opinions and analysis on the strengths and limitations of the measures and  any potential options identified. We also asked the stakeholders to discuss  the validity and reliability of the measures and any potential  improvements.
Although representatives from the think tanks and academicians we  interviewed identified limitations with the data that are used to make  international comparisons, stakeholders generally use the same sources,  thought the data were adequate, and support current efforts being made to  improve the quality of the data.
We conducted this performance audit from February 2009 through  October 2009 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
section_title: Appendix II: Comments from the Department of Commerce
section_title: Appendix III: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, David Sausville (Assistant  Director), Eli Albagli, Derrick Collins, Amy Rosewarne, Andrew Stavisky,  Hai Tran, Amy Ward-Meier, and Mindi Weisenbloom made key  contributions to this report.