Public-Safety Broadband Network: FirstNet Should Strengthen Internal Controls and Evaluate Lessons Learned
Released: 2015-04-28T00:00:00


Highlights: section_title: Why GAO Did This Study
paragraphs: For communications during emergencies, public safety officials rely on thousands of separate systems, which often lack interoperability, or the ability to communicate across agencies and jurisdictions. The 2012 act created FirstNet within the Department of Commerce to establish, for public safety use, a nationwide, interoperable, wireless broadband network, which will initially support data transmissions. The 2012 act established numerous responsibilities for FirstNet, provided $7 billion for network construction, and required FirstNet to be self-funding beyond this initial allocation. As part of the effort, FirstNet is working with five “early builder projects” that are building local and regional public-safety broadband networks.

GAO was asked to examine FirstNet's progress in establishing the network. GAO assessed (1) FirstNet's progress carrying out its responsibilities and establishing internal controls, (2) how much the network is estimated to cost and how FirstNet plans to become self-funding, and (3) what lessons can be learned from the early builder projects. GAO reviewed FirstNet documentation and public-safety network cost estimates, surveyed all state-designated FirstNet contacts, and interviewed FirstNet officials and public safety stakeholders selected for their telecommunications and public safety experience.

section_title: What GAO Found
paragraphs: The First Responder Network Authority (FirstNet) has made progress carrying out its responsibilities established in the Middle Class Tax Relief and Job Creation Act of 2012 (the 2012 act) but lacks certain elements of effective internal controls. FirstNet is charged with the complex and challenging task of establishing a new, nationwide, wireless broadband network for public safety entities, in consultation with federal, state, local, and tribal stakeholders. The network will initially support interoperable data communications, and later integrate mission-critical voice capabilities as public safety standards for voice communications are developed. FirstNet has made progress establishing an organizational structure, planning for the network, and consulting with stakeholders. FirstNet has also begun establishing policies and practices consistent with federal internal control standards. Officials told GAO that they plan to continue to do so. However, FirstNet has not fully assessed its risks or established standards of conduct—which is an important form of ethical guidance for its personnel . Given that FirstNet faces numerous risks to achieve its complex objectives, fully assessing risks could help FirstNet achieve its objectives and maximize use of its resources. Developing standards of conduct could also help FirstNet address any performance issues in a timely manner.

A nationwide public-safety broadband network has been estimated by various entities to cost billions of dollars, and FirstNet faces difficult decisions determining how to fund the network's construction and ongoing operations. These estimates indicate the cost to construct and operate such a network could be from $12 to $47 billion over the first 10 years. The actual cost of FirstNet's network will be influenced by FirstNet's (1) business model, especially the extent of commercial partnerships; (2) use of existing infrastructure; (3) efforts to ensure network reliability; and (4) network coverage. For example, the cost of the network may be higher if FirstNet does not utilize partnerships and some existing infrastructure. To become self-funding, FirstNet is authorized to generate revenue through user fees and commercial partnerships. However, FirstNet faces difficult decisions in determining how to best utilize these revenue sources. For instance, widespread network coverage can attract more users and revenue, but is expensive to construct and maintain, especially in rural areas.

FirstNet has taken steps to collect and evaluate information and lessons from the five “early builder projects” that are developing local and regional public-safety networks, but could do more to ensure that the lessons are properly evaluated. For example, FirstNet has asked the projects to report on the experiences of their networks' users and has assigned contractors to collect and log lessons. However, FirstNet does not have a plan that clearly articulates how it will evaluate those experiences and lessons. Although FirstNet told GAO that it remains in close contact with early builder projects, GAO has previously found that a well-developed evaluation plan for projects like these can help ensure that agencies obtain the information necessary to make effective program and policy decisions. Given that the early builder projects are doing on a local and regional level what FirstNet must eventually do nationally, an evaluation plan can play a key role in FirstNet's strategic planning and program management, providing feedback on both program design and execution and ensuring FirstNet has not missed opportunities to incorporate lessons the projects have identified.

section_title: What GAO Recommends
paragraphs: FirstNet should complete its risk assessment, develop standards of conduct, and develop an evaluation plan for early builder projects. FirstNet concurred with the recommendations.

section_title: Letter
paragraphs: Communication systems are essential for public safety officials— especially first responders such as police, firefighters, and paramedics— to gather and share information during emergencies. First responders rely  on thousands of separate, incompatible, and often proprietary land-mobile  radio (LMR) systems for their mission-critical voice communications.  Oftentimes these LMR systems lack “interoperability”—the capabilities  that allow first responders to communicate with their counterparts in other  agencies and jurisdictions with differing systems—which has been a long- standing concern. For example, during the terrorist attacks of September  11, 2001, and also during Hurricane Katrina in 2005, the lack of  interoperable public-safety communications hampered rescue efforts and  the overall effectiveness of public safety operations.

The Middle Class Tax Relief and Job Creation Act of 2012 provides  resources, including both spectrum and billions of dollars, for the  establishment of a nationwide, interoperable wireless broadband network  specifically for first responders. In particular, the 2012 act created the  First Responder Network Authority (FirstNet) as an independent authority  within the Department of Commerce’s National Telecommunications and  Information Administration (NTIA) to establish a nationwide public-safety   allocated billions for its construction, and set aside  broadband network,radio frequency spectrum on which the network can operate. The  network is expected to support important data transmission (such as the  vital signs of critically injured people and security-camera video feeds)  and foster greater interoperability among public safety entities. Unlike  current LMR systems, the devices operating on the public-safety  broadband network are expected to be interoperable among first  responders using the network, since they are using the same radio  frequency band nationwide and will be required to be built to the same  open, non-proprietary, commercially available standards.will initially support data transmissions and non-mission critical voice, with  mission-critical voice communications expected to be integrated in the  coming years.

In developing the public-safety broadband network, FirstNet is faced with  a complex and challenging mission. We previously reported on the  challenges to implementing a public-safety broadband network, including  ensuring the network’s reliability and security and obtaining adequate  funds to build and maintain it. To inform its work, FirstNet is required to  consult with numerous federal, state, local, and tribal jurisdictions.  FirstNet is also working with five “early builder projects” that received  federal funding to deploy local and regional public-safety broadband  networks similar to what FirstNet is required to establish on a national  scale. FirstNet also has to develop a business plan that supports the  upfront and ongoing costs of the public-safety broadband network.

You asked us to examine FirstNet’s progress in establishing the  nationwide public-safety broadband network. This report assesses (1) the  extent to which FirstNet is carrying out its responsibilities and establishing  internal controls for developing the public safety network, (2) how much  the public safety network is estimated to cost to construct and operate  and how FirstNet plans to become a self-funding entity, and (3) what  lessons can be learned from local and regional public-safety network  early builder projects.

To address these objectives, we reviewed FirstNet’s documentation, such  as Requests for Information (RFI), annual reports to Congress, board- meeting materials and resolutions, and presentations to stakeholders. We  compared FirstNet’s efforts and progress to carry out its responsibilities  with requirements established in the Middle Class Tax Relief and Job  Creation Act of 2012 and compared FirstNet’s efforts to establish internal  controls with criteria established in the federal Standards for Internal  Control. We reviewed cost estimates for a nationwide public-safety  broadband network from a variety of entities, including the Congressional  Budget Office, the Federal Communications Commission (FCC), and  academics. We also reviewed FirstNet’s cost estimate and conducted  interviews with FirstNet officials to learn more about how the estimate  was prepared, the assumptions underlying it, and the documentation  supporting it. We assessed FirstNet’s cost estimate against the criteria  established in our Cost Guide. We also interviewed early builder project  officials from—and involved in—the five current projects (Los Angeles,  CA; Adams County, CO; New Jersey; New Mexico; and Harris County,  TX), three projects that were canceled (Charlotte, NC; Mississippi; and  San Francisco, CA), and state and local public-safety entities in the  project jurisdictions. We reviewed the Spectrum Manager Lease  Agreements (SMLA) that each of the five current projects established with  FirstNet, and documentation related to how FirstNet plans to collect  lessons learned from the projects. We assessed FirstNet’s plans to  evaluate and utilize lessons from the projects against key features of a  well-developed evaluation plan for pilot projects identified by our previous  reports.

To obtain stakeholder views, we surveyed all 50 states, the District of  Columbia, and 5 U.S. territories. We administered the survey from  October 2014 through November 2014 and received 55 responses, for a  98 percent response rate. and a variety of other stakeholders, such as state and local public-safety  entities, commercial wireless carriers, subject matter experts, public  safety associations, and federal agencies including FCC and the  Department of Homeland Security (DHS). We also spoke with  government officials in Sweden responsible for establishing a public- safety communications network in their country. We selected  stakeholders by considering their involvement in the early builder project  jurisdictions, experience with operating and using wireless  communications systems and public-safety communications systems and  devices in particular, familiarity with FirstNet and its mission, and—to  obtain a cross-section of public safety disciplines—their public safety  role. More details about our scope and methodology can be found in  appendix I and a copy of our survey in appendix II.

We interviewed FirstNet and NTIA officials  We conducted this performance audit from April 2014 to April 2015 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

Puerto Rico did not respond to our survey.
subsections: section_title: Background
paragraphs: The Middle Class Tax Relief and Job Creation Act of 2012 (2012 act)  established numerous responsibilities for FirstNet, most of which relate  directly to developing the nationwide public-safety broadband network  (hereafter, public safety network). It is intended that this network will be a  new, nationwide, broadband telecommunications network. Public safety  users, and potentially other “secondary” users, may be assessed fees to  use the network, much as they may currently pay for the use of  commercial wireless networks. In establishing this network, FirstNet must  issue open, transparent, and competitive Requests for Proposals  (RFP) to private sector entities for the purpose of building, operating,  and maintaining the public safety network; develop RFPs that include appropriate timetables for construction,  and network coverage areas and service levels, among other things; enter into agreements to use, to the maximum extent economically  desirable, existing commercial, federal, state, local, and tribal  infrastructure; promote competition in the public-safety equipment marketplace by  requiring that equipment for the public safety network be built to open,  non-proprietary standards; promote integration of the public safety network with public-safety address special considerations for areas with unique homeland or  require deployment phases that include substantial rural coverage  milestones; and develop the technical and operational requirements for the public  safety network, as well as the practices and procedures for managing  and operating it.

In establishing the infrastructure for the public safety network, the 2012  act requires FirstNet to include the following network components, as  shown in figure 1:  core network of data centers and other elements, all based on  commercial standards;  connectivity between the radio access network (RAN) and the public  Internet or the Public Switched Telephone Network, or both; and network cell-site equipment, antennas, and “backhaul” equipment,  based on commercial standards, to support wireless devices  operating on frequencies designated for public safety broadband.

In developing the public safety network, FirstNet must work with a variety  of stakeholders. The 2012 act required FirstNet to be headed by a 15- member board with 3 permanent members (the Secretary of Homeland  Security, the Attorney General, and the Director of the Office of  Management and Budget) and 12 individuals appointed by the Secretary  of Commerce. The appointed members are required to have public safety  experience or technical, network, or financial expertise. The 2012 act  also required FirstNet to establish a standing public-safety advisory  committee to assist it in carrying out its responsibilities and consult with  federal, regional, state, local, and tribal jurisdictions on developing the  network. For state, local, and tribal planning consultations, FirstNet is  required to work with the state Single Points of Contact (SPOC) who have  been designated by each state, territory, and the District of Columbia  (hereafter, states).

The 2012 act requires FirstNet to notify the states when it has completed  its RFP process for building, operating, and maintaining the public safety  network. Once a state receives the details of FirstNet’s plans, it has 90  days either to agree to allow FirstNet to construct a RAN in that state or  notify FirstNet, NTIA, and FCC of its intent to deploy its own RAN. A state  that “opts out” of FirstNet’s plan to build that state’s RAN has an  additional 180 days from that notification to develop and complete RFPs  for the construction, maintenance, and operation of the RAN within the  state; the state must send FCC an alternative plan for constructing,  operating, and maintaining its RAN. The plan must demonstrate that the  state’s proposed RAN would comply with certain minimum technical  requirements and be interoperable with FirstNet’s network. FCC shall  either approve or disapprove the plan. In addition, a state that opts out is  required to apply to NTIA for an agreement to use FirstNet’s spectrum  capacity.

Various federal agencies will provide assistance and support to FirstNet.  For example,

The National Institute of Standards and Technology (NIST) within the  Department of Commerce, in consultation with FCC, DHS, and the  Department of Justice’s National Institute of Justice, is required to  conduct research and assist with the development of standards,  technologies, and applications to advance wireless public-safety  communications.

The Public Safety Communications Research (PSCR) program, a joint  NTIA and NIST effort, performs research on behalf of FirstNet to  advance public-safety communications interoperability.

DHS, through both the Office of Emergency Communications and the  Office for Interoperability and Compatibility, supports the  establishment of the public safety network and collaborates with  public safety and government officials at the federal, state, local, and  tribal levels to help ensure the network meets the needs and technical  requirements of users in the public safety community.

The 2012 act also required FCC, the entity responsible for managing and  licensing commercial and non-federal spectrum use—including spectrum  allocated to public safety—to take certain steps to support FirstNet. It also  authorized FCC to provide FirstNet with technical assistance and to take  any action necessary to assist FirstNet in effectuating its statutory duties  and responsibilities. Radio frequency spectrum is an essential resource  for wireless communications, including the planned public-safety network.  The energy in electronic telecommunications transmissions converts  airwaves into signals to deliver voice, text, and images. These signal  frequencies are allocated for specific purposes, such as television  broadcasting or Wi-Fi, and assigned to specific users through licenses.

Allocating sufficient spectrum for wireless emergency communications  has long been a concern for Congress. The 2012 act required FCC to  reallocate the “D Block,” a previously commercial spectrum block located  in the upper 700 megahertz (MHz) band, to public safety and to grant a  license to FirstNet for the use of both the existing public-safety broadband  spectrum in the upper 700 MHz band and the D Block. FirstNet now has  a license to operate the nationwide public-safety broadband network on  spectrum in the upper 700 MHz band, specifically 758-769 MHz and 788- 799 MHz (see fig. 2 below). In October 2013, FCC adopted consolidated  technical service rules governing the 700 MHz band of spectrum licensed  to FirstNet and was prepared to accept and process applications for  equipment certification in that spectrum band consistent with the newly  consolidated rules. FCC has conducted specified spectrum auctions, as  required by the 2012 act, so that auction proceeds could be used to fund  FirstNet. The 2012 act provides $7 billion from the spectrum auction  proceeds to FirstNet for “buildout” of the public safety network, reduced  by the amount needed to establish FirstNet. The 2012 act requires  FirstNet to be self-funding beyond this initial $7 billion.

Efforts to establish local and regional public-safety networks are also  ongoing, and predate the 2012 act. In 2009, public safety entities began  requesting waivers from FCC’s rules to allow early deployment of  interoperable public-safety wireless broadband networks in the upper 700  MHz spectrum band authorized for public safety use. Through 2011, FCC  granted waivers to 22 jurisdictions for early deployment of such  networks.altered the regulatory landscape for the 700 MHz band” and sought   However, FCC recognized that the 2012 act “fundamentally  comment on the disposition of waiver jurisdiction deployments. In May  2012, NTIA partially suspended seven grant projects for the waiver  jurisdictions that received funding from the Broadband Technology  Opportunities Program (BTOP), a federal grant program to promote the  expansion of broadband infrastructure. NTIA suspended the projects in  part to ensure the projects supported the development of FirstNet’s public  safety network. In July 2012, FCC decided to hold all waiver  authorizations ineffective as of September 2, 2012, and said it would  allow such jurisdictions to continue to deploy public-safety broadband  services in the existing public-safety broadband spectrum (later extended  to include the D Block) under certain limited circumstances consistent  with FirstNet’s future use of the spectrum. FCC granted FirstNet the 700  MHz public-safety broadband-spectrum license in November 2012, and  in February 2013, FirstNet announced it would begin negotiations with the  original waiver jurisdictions on SMLAs. With FirstNet in control of the 700  MHz public-safety broadband-spectrum license pursuant to its FCC  license, the jurisdictions had to secure an SMLA with FirstNet to lift their  partial funding suspension and resume work. By August 2014, FirstNet  secured SMLAs with five original waiver jurisdictions; in this report, we  refer to these jurisdictions as early builder projects. Figure 3 shows  timelines for the early builder projects and the amount of federal funding  they received to deploy their networks. As part of their SMLAs with  FirstNet, the early builder projects agreed to report to FirstNet on their  use of the spectrum, their progress, and on key-learning conditions,  among other things. FirstNet has not yet determined if or how the early  builder project networks will be incorporated into its nationwide network.

Three other original waiver jurisdictions were awarded BTOP funds to  deploy local and regional public-safety broadband networks, including the  State of Mississippi,  the San Francisco Bay Area in California, and the  City of Charlotte in North Carolina. However, for various reasons, these  jurisdictions were unable to reach SMLAs with FirstNet and the projects  were canceled.

Even with the establishment of the public safety network, first responders  will continue to rely on their current LMR systems for mission-critical voice  communications. We previously reported that a major limitation of a  public-safety broadband network is that it will not provide first responders  with “mission-critical” voice capabilities—that is, voice capabilities that  meet a high standard for reliability, redundancy, capacity, and flexibility  necessary during emergencies—for several years. Long Term Evolution (LTE), the technical standard for the public-safety broadband network, is  a wireless broadband standard that is not currently designed to support  mission-critical voice communications.are currently providing voice over LTE capabilities, but these capabilities  do not currently meet public safety’s mission-critical voice requirements  because key elements needed for mission-critical voice—such as group  calls—are not part of the LTE standard. Therefore, public safety agencies  will continue to rely on their LMR systems for mission-critical voice for the  foreseeable future, and the public safety network will supplement, rather  than replace, current LMR systems.

section_title: FirstNet Is Making Progress Meeting Responsibilities but Lacks Certain Elements of Effective Internal Controls
subsections: section_title: FirstNet Is Making Progress Carrying Out Statutory Responsibilities
paragraphs: FirstNet has made progress carrying out its statutory responsibilities in  three areas—(1) establishing its organizational structure, (2) planning the  public safety network, and (3) consulting with stakeholders—but could  face challenges in each of these areas. Figure 4 provides a timeline of  select events in FirstNet’s progress meeting its statutory responsibilities  since the 2012 act’s passage.

As a newly created entity within the federal government, FirstNet has  taken a number of steps to establish its organizational structure and hire  staff. As required by the 2012 act, the Secretary of Commerce appointed  FirstNet’s inaugural board members in August 2012. FirstNet’s Board  established Governance and Personnel, Finance, Technology, and  Outreach Committees to review, approve, and oversee FirstNet’s  activities. As required by the 2012 act, FirstNet’s Board also established  the Public Safety Advisory Committee (PSAC) in February 2013—one  year after the 2012 act’s passage—and adopted the PSAC’s charter in  June 2014.

The PSAC’s membership represents a broad cross section  of public safety disciplines and state, local, and tribal governments. In  addition to the board and the PSAC, FirstNet has established  organizational units and hired key personnel to lead and perform its work.  In April 2013—over a year after the 2012 act’s passage—the FirstNet  Board selected an Executive Director to lead its day-to-day operations.  Since then, FirstNet has hired, and continues to hire, other senior  management personnel to lead its organizational units (such as a Chief  Counsel and Chief Administrative, Financial, and Information Officers),  and Directors and organizational chiefs to further lead and perform its  work. FirstNet has also hired, and continues to hire, general staff. As of  February 2015, FirstNet had over 120 employees, including full-time  equivalent federal employees, contractors, and personnel on detail from  other agencies. FirstNet confirmed in February 2015 that it may hire up to  42 additional full-time equivalent federal employees in fiscal year 2015  and that it will evaluate and adjust needed staffing levels on an ongoing  basis. See figure 5 for an overview of FirstNet’s organizational structure.  Additionally, in June 2014, FirstNet opened its “corporate” headquarters  in Reston, Virginia, and, in March 2014, a technical center to serve as its  technical, engineering, and network design headquarters in Boulder,  Colorado.

Stakeholders we spoke with and surveyed expressed concern that  organizational issues have slowed FirstNet’s progress and could continue  to do so. In particular, in response to our survey, numerous SPOCs noted  either that FirstNet’s placement within NTIA could create “bureaucratic”  obstacles or that FirstNet should be more independent from NTIA. One  survey respondent and another subject matter expert we interviewed also  noted that this placement creates conflicting expectations—that is,  stakeholders expect FirstNet to behave like both a commercial wireless  carrier and a government entity, and these expectations can sometimes  be in conflict. However, FirstNet officials told us that while FirstNet has  leveraged its relationship with NTIA in administrative and legal matters, it  exercises strong independence in decisions that are directly program- related.

Numerous stakeholders we surveyed and interviewed were also  concerned about the pace of FirstNet’s hiring, noting that the federal  hiring process is too slow, has not allowed FirstNet to hire staff quickly  enough, and has delayed FirstNet’s progress. For example, FirstNet hired  its management team for state plans, consultation, and outreach in late  2013 to early 2014, then hired State and Local, Tribal, and Federal  Outreach Leads in June and August 2014, almost 2 years after FirstNet’s  board members were appointed. Additionally, FirstNet is still in the  process of hiring staff for key positions. For example, as of February 2015  FirstNet was re-filling the Chief User Advocacy Officer and Director of  Communications positions, was planning for the hiring of a Chief  Procurement Officer, and was in the process of hiring staff to lead  regional consultation teams that it plans to establish, among other  positions. As of February 2015, it was also still hiring key technical  positions—including Director of Devices, Director of Standards, and  Director of Core Network—and re-filling the Chief Technology Officer  position. According to FirstNet officials, FirstNet faces challenges hiring  as quickly as it would like, in part due to government hiring procedures,  but is seeking direct hire authority from the Office of Personnel  Management and is exploring other authorities it could use to expedite  hiring.
subsections: section_title: Planning the Public Safety Network
paragraphs: To plan the public safety network and help ensure that its approach is  open and transparent and meets the 2012 act’s requirements and that  interested parties have a formal way to comment on FirstNet’s strategy  and decisions, FirstNet intends to follow the Federal Acquisition  Regulation (FAR) process for its comprehensive network-services  procurement. This process will culminate in one or multiple RFPs for  “network solution(s)”—that is, proposals for the building, deployment,  operation, and maintenance of the public safety network. To help draft the  RFP(s), FirstNet has sought comments through a Notice of Inquiry (NOI),  RFIs, and Public Notices. Specifically, in October 2012, NTIA, on behalf  of FirstNet, issued an NOI to seek input on the network’s potential  architecture.RFIs on specific technical aspects of the network and on devices and   From April to November 2013, FirstNet issued 12 detailed  applications for public safety. NTIA and FirstNet received over 400  comments to the NOI and RFIs, and FirstNet has used these as a source  of market research in assessing industry capabilities and developing the  technical design of the network. Building off these efforts, in September  2014 FirstNet issued an RFI that sought further comment on a number of  issues, such as network coverage, pricing, deployment strategies,  security, prioritization among network users, and customer service,  among other things. According to this RFI, FirstNet’s current approach is  to follow a performance-based procurement strategy, whereby FirstNet  does not dictate the specific network solution that bidders responding to  the RFP(s) must provide, but, rather, outlines objectives and encourages  bidders to develop proposals that will meet those objectives.  Additionally, although the 2012 act exempts FirstNet from portions of the  Administrative Procedure Act (APA), FirstNet has chosen to follow an  APA-like process in order to provide more opportunities for interested  parties to comment on its interpretations of the 2012 act’s requirements.  Therefore, in September 2014, FirstNet also issued a Public Notice  seeking comment on a number of its preliminary interpretations, such as how to define its rural coverage milestones and eligible network users.FirstNet received 185 responses to the September 2014 RFI and Public  Notice from a broad range of respondents, such as states, public safety  entities, private companies, and associations. FirstNet issued a second  Public Notice in March 2015 and expects to issue a draft RFP in spring  2015.

In addition to its efforts seeking public comment, FirstNet has also  received technical input from a variety of stakeholders, such as FCC, the  PSAC, the National Public Safety Telecommunications Council (NPSTC),  and the PSCR program. As required by the 2012 act, the FCC’s Technical Advisory Board for First Responder Interoperability issued  minimum technical requirements for the public safety network in May  2012. The report provided recommendations on LTE standards; network  user equipment, device management, and evolution; and quality of  service, among other things. FirstNet has also asked the PSAC to provide  recommendations on various topics, including a framework for  establishing the priority of public safety entities on the network. In May  2014, NPSTC provided FirstNet with guidance on what makes  communication systems “public-safety grade.” Additionally, FirstNet has  provided the PSCR—which is based in Boulder, Colorado, where  FirstNet’s technical center is also located—with funding to research  interoperability standards, test and evaluate potential network features,  and model and simulate network data traffic.

Specifically, SPOCs responded this 33 percent of the time. For question wording, see  appendix II question 5 (b-g). network, and the practices, procedures, and standards for managing and  operating the network. Still, even in these instances, SPOCs’ level of  satisfaction or dissatisfaction was mostly “moderate.” While many  stakeholders we surveyed and interviewed noted that FirstNet’s progress  has been too slow, some also noted that they were satisfied with  FirstNet’s progress given the complex nature of FirstNet’s tasks and that  it is a government entity subject to federal rules and regulations. Some  also said that FirstNet’s progress has improved recently, especially as  FirstNet has hired more staff.

section_title: Consulting Stakeholders
paragraphs: FirstNet has initiated a process to consult with the public safety  community in each state through the SPOC. FirstNet first worked to  establish informal mechanisms for coordinating with the SPOCs. For  example, FirstNet began holding regional workshops in May 2013,  quarterly SPOC webinars in January 2014, and monthly calls with SPOCs  in February 2014. FirstNet began its formal state consultation process by  delivering an initial consultation package to each SPOC in April 2014; the  package contained a checklist for the SPOCs to complete in preparation  for an initial consultation meeting. FirstNet is using these meetings to  gather information on each state’s unique challenges, needs, and  processes to inform its development of the public safety network and, in  particular, state RAN plans. Initial state consultation meetings began in  July 2014—when FirstNet conducted its first consultation with Maryland— and FirstNet expects these initial meetings to continue through 2015, with  additional rounds to follow. As of April 16, 2015, FirstNet had conducted  initial consultation meetings with 18 states, Puerto Rico, and the District  of Columbia.

In addition to its consultation with SPOCs, FirstNet officials have  conducted outreach to other stakeholders. For example, from October  2013 to February 2015, FirstNet officials visited 39 states and territories  while participating in 187 events, such as state town halls and public  safety, industry, and government (including federal, state, local, and  tribal) conferences. To engage with tribal entities, FirstNet staff visited  eight tribal nations in 2014, and sought the advice of the PSAC on tribal  outreach, education, and inclusive consultation strategies. To engage  with federal entities, FirstNet has hired a director of federal outreach,  designated DHS’s Emergency Communications Preparedness Center  (ECPC) as the primary body to FirstNet for federal consultation, and  participated in meetings with ECPC and other agencies. To engage with  vendors interested in doing business with FirstNet, FirstNet has held  vendor meetings and appointed an industry liaison to coordinate vendor  outreach. In addition to this targeted outreach, FirstNet also launched a  public website in March 2014, where it regularly posts updates,  presentations, board-meeting minutes, a list of upcoming speaking  engagements, and other information. The 2012 act also created a State  and Local Implementation Grant Program (SLIGP), administered by NTIA,  to provide states with funds to plan for the public safety network and to  consult with FirstNet. Starting in July 2013, NTIA awarded $116 million  in SLIGP grants.

We believe FirstNet’s consultation and outreach activities generally align  with core principles for effective stakeholder participation that we have  developed and used in previous reports. For example, we found during the course of our review that FirstNet is using an open and clearly defined  decision-making process, actively conducting outreach, involving  stakeholders throughout the process, using formal and informal  participation methods, and including all stakeholders.

The majority of stakeholders we surveyed were generally satisfied with  the level of FirstNet’s consultation and outreach, but others were  dissatisfied and said that they would like more new information. In  response to our survey, 54 percent of SPOCs said they were either  “moderately” or “very” satisfied with FirstNet’s overall level of consultation,  coordination, and communication, while 22 percent said they were either  “moderately” or “very” dissatisfied. However, numerous stakeholders we  surveyed and interviewed said that they would like more new and detailed  information and that they would like FirstNet to focus more on certain  aspects of the public safety network during outreach. For example, in  response to our survey, SPOCs most frequently indicated that there  should be a “large increase” in how much FirstNet focuses on various  technical aspects of developing the network in its state outreach.SPOCs also said that the lack of new information from FirstNet, such as  details about the network’s design, hampers their ability to conduct local  outreach. However, in response to our survey many SPOCs also  acknowledged that the level of new and detailed information exchanged  will likely increase once they hold their initial state consultation meeting  with FirstNet. Additionally, officials from one federal agency with public  safety responsibilities said they would have liked to see more in-depth  outreach to federal entities, given that federal agencies will have a large  pool of potential network users. In January 2015, FirstNet formally  initiated its federal consultation process. As part of this process, FirstNet  will conduct meetings with federal entities in 2015 similar to the initial  consultation meetings it is conducting with states.

section_title: FirstNet Has Not Assessed Risk or Established a Complete Control Environment
paragraphs: Internal controls are the plans, methods, policies, and procedures that an  entity uses to fulfill its mission, strategic plan, goals, and objectives. An  effective internal control system increases the likelihood that an entity will  achieve its objectives. We assessed FirstNet’s policies and practices  against two components of an effective federal internal control system:  risk assessment and control environment.two components because risk assessment provides the basis for  developing appropriate risk responses and control activities and the  control environment is the foundation for an internal control system. For  those two components, we found that—although FirstNet has begun  establishing policies and practices that are consistent with federal  standards—FirstNet lacked certain elements that contribute to the proper  implementation of effective internal control systems. While FirstNet has  stated that it is relying on the Department of Commerce’s and NTIA’s  internal controls where it has not developed its own, it is also important  for FirstNet to implement its own controls, as the Commerce Office of  Inspector General (OIG) noted as early as February 2014 in a memo on  the management challenges facing FirstNet.
subsections: section_title: Risk Assessment
paragraphs: In an internal control system, according to federal internal control  standards, management should assess risks facing the entity as it seeks  to achieve its objectives. Specifically, entities should first clearly define  their objectives then identify and analyze risks from both internal and  external sources. Analyzing risks generally includes estimating the risk’s  significance, assessing the likelihood of its occurrence, and deciding how  to respond to it. Risk assessments inform an entity’s policies, planning,  and priorities, and help entities develop responses to the risks they face,  so that they can achieve their objectives. Control activities respond to  these risks.

FirstNet has set objectives and taken some steps to assess risks.  Specifically, FirstNet has set three key objectives:  1.  Provide FirstNet services that are critical to public safety users and  differentiate FirstNet services from commercial broadband services,  such as through reliability, resiliency, coverage, functionality,  interoperability, quality of service, priority access, pre-emption, and  applications.  2.  Reduce costs for public safety entities by leveraging the value of  excess network capacity with partners.  3.  Provide mechanisms for public safety entities (directly or indirectly  through the states) to benefit from the economies of scale created by  FirstNet in terms of purchasing, partnering, and information/data.

FirstNet has further delineated how it will accomplish these objectives in a  “roadmap” that identifies additional long-term and short-term objectives  and milestones. FirstNet has established a Program Management Office  to, according to FirstNet officials, help set internal timelines and monitor  the completion of tasks needed to achieve these objectives and reach  these milestones. Additionally, the Department of Commerce and FirstNet  have performed some risk assessment activities. In February 2014, the  Commerce OIG issued a memorandum outlining the top management  challenges facing FirstNet. To support development of the roadmap,  FirstNet created a “risk register” that identifies some risks related to its  financial sustainability as well as possible counter-measures.

However, FirstNet has not yet fully assessed risks it may face in  accomplishing its objectives. FirstNet officials told us in November 2014  that they have not yet done so because they are in the process of  defining risk factors and, again in December 2014, because they are in  the process of conducting a legal compliance risk assessment of certain  key risk areas. In December 2014, FirstNet officials also said that they  intend to perform periodic risk assessments in various areas to manage  and mitigate risks on an iterative basis. However, as of February 2015, FirstNet has not yet completed these risk assessment activities. As a  result, we were unable to evaluate the extent to which these activities  align with the elements of risk assessment detailed in the federal internal  control standards, and therefore, it remains unclear how effective  FirstNet’s efforts will be in helping it to identify and respond to obstacles  to fulfilling its responsibilities.

As FirstNet completes these assessments, we believe that it is important  that it incorporate all of the elements of risk assessment detailed in the  federal internal control standards. Lacking complete risk assessments  (that is, assessments that incorporate these elements), FirstNet’s control  activities may not be designed to respond to the appropriate risks. In  previous work, we found that when an agency worked quickly to establish  a new program, it resulted in the agency’s hastily designing an internal  control system that was not based on complete risk assessments and that  the agency responded to risks in a reactive, rather than a proactive,  manner. Thus, it was unclear whether the controls appropriately  responded to risk or were the best use of the agency’s resources.  Additionally, we are concerned that the complexity of FirstNet’s  responsibilities and objectives, makeup of FirstNet’s Board of Directors,  and challenges that FirstNet will face attracting users to its network and  becoming self-funding illustrate the multitude of potential risks FirstNet  faces in achieving its objectives. For example, as we point out later in this  report, various factors could hinder whether public safety entities adopt  the public safety network—and thus how much user fee revenue FirstNet  can collect—which could pose risks to FirstNet’s ability to become self- funding. Given this, complete risk assessments could help FirstNet  appropriately design its full internal control system and achieve its  objectives while maximizing use of its available resources.

section_title: Control Environment
paragraphs: The control environment is the foundation for an internal control system  and provides the basic structure that helps an entity achieve its  objectives. To help set this environment, according to federal internal  control standards, an entity should establish an organizational structure and delegate authority establish appropriate human-capital practices for hiring, developing,  evaluating, retaining, supervising, and disciplining personnel; demonstrate a commitment to competence, such as by establishing  expectations of competence and holding personnel accountable by  evaluating their performance; demonstrate a commitment to integrity and ethical values, such as by  setting a positive “tone at the top,” providing and evaluating  adherence to ethical and behavioral guidance, and removing  temptations for unethical behavior; and have an oversight body that oversees the entity’s internal control  system.

FirstNet has taken a number of steps to begin establishing an effective  control environment. As described above, FirstNet has established an  organizational structure with clearly designated responsibilities and has  explored hiring options that would allow it to recruit individuals more  quickly. According to FirstNet officials, to develop staff, FirstNet has also  leveraged its relationship with the Department of Commerce by using the  Department’s training facilities as it works to create FirstNet-specific  training. Per Department of Commerce policy, FirstNet’s Senior Executive  Service employees are subject to applicable competency and evaluation  plans and other employees are evaluated using a performance appraisal  process. Additionally, FirstNet has taken steps to identify and manage  potential conflicts of interest. For example, FirstNet has held ethics  briefings, distributed ethics documents, held ethics counseling and  training, and instituted a Board Member Vendor Interaction Policy to  establish processes for board members interacting with vendors with a  potential interest in FirstNet’s procurement efforts. FirstNet has also  established and disseminated a variety of guidance documents, such as  policies on employee timekeeping, expenses, travel, information  technology rules of behavior, and telework. Finally, FirstNet’s Board  Committees and FirstNet senior management’s Compliance Committee  oversee FirstNet’s activities and the Commerce OIG and others perform  additional oversight.

However, FirstNet has not yet finished establishing its control  environment. While FirstNet has a variety of separate guidelines and  policies, it does not have a uniform and cohesive standards of conduct  policy. Specifically, although FirstNet officials told us that they intend to  develop a code/standards of conduct policy, which is an important form of  ethical and behavioral guidance for personnel, they have not yet  developed this item as of February 2015. According to FirstNet officials,  FirstNet has not yet done so because, as a “start-up” entity, building up  the organization while making progress meeting statutory responsibilities  is a balancing act affected by FirstNet’s priorities and resources.

Nonetheless, absent standards of conduct, we are concerned that  FirstNet may not be able to address deviations in its personnel’s conduct  and performance, and take corrective actions in a timely manner. Indeed,  FirstNet itself established a special committee in May 2013 to review  ethical concerns raised by one of its board members. Similarly, in a  December 2014 report, the Commerce OIG identified concerns with  FirstNet’s financial disclosure reporting and contracting practices, among  other things. The report highlighted that the FirstNet Board, out of  necessity, includes members with significant ties to the  telecommunications industry that make strategic decisions regarding  FirstNet’s operations and, thus, are at increased risk of encountering  conflicts of interests. Although FirstNet has taken corrective actions since  the Commerce OIG’s investigation, we believe that establishing and  evaluating adherence to standards of conduct may help FirstNet ensure  that all its personnel are held accountable for their actions. By  establishing this item, FirstNet could also foster stakeholder trust in its  ability to meet its statutory responsibilities and be a good steward of  public funds.

section_title: FirstNet Faces Difficult Decisions in Determining How to Pay for a Nationwide Public-Safety Network Estimated to Cost Billions
subsections: section_title: Cost of a Public Safety Network Estimated to be at least $12 Billion over First 10 Years
paragraphs: Various entities have estimated the cost to construct and operate a  nationwide network for public safety from a low of $12 billion to a high of  between $34 and $47 billion, over the first 10 years. As shown in table 1,  a variety of entities have developed cost estimates for a public-safety  broadband network, although they have used different assumptions about  the network’s scope. Key assumptions influencing these estimates  include whether the network is constructed, operated, or financed in  partnership with commercial entities, and the number of sites needed to  provide the network’s coverage. For instance, FCC’s 2010 estimate  assumes costs would be decreased through a high level of cooperation  with commercial carriers. These estimates also vary, for instance, on how  much they expect ongoing maintenance and operation to cost. However,  some differences among these estimates are difficult to identify since  some of the estimates do not explicitly state all their assumptions.

The actual costs per site for early builder projects vary but are generally  less than the estimates above. As discussed more below, five early  builders are constructing local or regional broadband networks for public  safety. Some of these projects have begun construction and provided us  with cost data. The new cell towers (and associated LTE equipment) for  the regional network in the Los Angeles area average $196,000 per site.  For equipment being attached to existing towers and sites, the average  cost is $102,000. For the regional project in Adams County, Colorado, the  cost per site is approximately $75,000. This cost reflects sites utilizing  existing infrastructure that does not require strengthening. For budgetary  purposes, the Adams County project has estimated the development of a  new site at $500,000, an estimate that includes acquisition of land, tower  construction, and utilities.

GAO-09-3SP. associated documentation were deemed business sensitive. Therefore,  we cannot say if the estimate is in line with the best practices associated  with the credible and accurate characteristics of our Cost Estimating and  Assessment Guide.

section_title: Various Factors Will Influence Cost of FirstNet’s Public Safety Network
paragraphs: Cost estimates notwithstanding, various factors will influence the cost of  constructing and operating FirstNet’s public safety network, including (1)  the business model used, especially the extent of commercial  partnerships; (2) use of existing infrastructure; (3) efforts to ensure  network reliability; and (4) network coverage.
subsections: section_title: Extent of Commercial Partnerships
paragraphs: FirstNet’s business model, especially the extent to which it partners with  commercial carriers or other private enterprises, will influence the cost to  construct and operate the public safety network. The 2012 act gives  FirstNet the authority to engage in a variety of commercial partnerships.  Such partnerships could involve a private-sector partner that would  contribute resources to the network (e.g., infrastructure) and accept some  risk in the form of profits or losses. FirstNet would then contribute other  resources to the partnership (e.g., spectrum) conditioned on the network  satisfying social objectives (i.e., enhancing public safety  communications). In a partnership, public safety and commercial users  could share the public safety network’s infrastructure and spectrum, with  public safety given priority to all network capacity during times of  emergency. Regardless of the approach, some public safety stakeholders  we spoke with maintained the need for FirstNet to work with commercial  partners in building and operating the network for it to be financially  sustainable. One study calculated that the value of serving both  commercial and public safety users is greater than the costs of the  additional capacity and signal reliability requirements placed on the  network to serve both public safety and commercial users, demonstrating  a strong business case for a public-private partnership.

section_title: Use of Existing Infrastructure
paragraphs: The use of existing infrastructure will influence the cost to construct and  operate FirstNet’s public safety network. Under agreements to share  existing wireless network infrastructure, FirstNet may be able to make use  of, for example, cell towers, antennae, cabling, radio-processing  equipment, backup power facilities, and the links between towers and the  nearest communications hub, to the extent economically desirable to do  so. According to FCC estimates, capital costs would be 2.5 times greater  without this form of sharing. Given these potential financial savings, a few  public safety stakeholders we spoke with maintained that FirstNet should  use at least some existing infrastructure for the network. The use of  existing infrastructure can have limitations though. For instance, using  existing infrastructure can limit the design and coverage of the network,  since existing towers and buildings can only facilitate certain network  coverage given their physical location. Negotiating access to existing  infrastructure can also be a time-consuming process, especially with  government-owned or controlled facilities, and where contracts must by  executed with multiple owners, ultimately slowing down network  deployment. For example, when we spoke with Swedish officials about  the public-safety communications network in their county, they said that  they sought to use existing infrastructure, to save costs, when  constructing their network, but faced problems in their largest cities  convincing tower owners to allow the government to rent the towers. Furthermore, there is a risk when public safety entities rely on  infrastructure owned by commercial operators, particularly if they have to  rely on a single provider in any given location that can then charge high  fees.

section_title: Efforts to Ensure Network Reliability
paragraphs: FirstNet’s approach to ensure the public safety network is safe, secure,  and resilient (that is, the overall reliability of the network) will also  influence the cost to construct the network. FirstNet is required by the  2012 act to “ensure the safety, security, and resiliency of the network.”

This interoperable network in Sweden is called the “Rakel” network and is primarily used  for voice communications.

Pub. L. No. 112-96, § 6206(b)(2)(A), 126 Stat. 156, 212.

FirstNet is still determining how it will satisfy this requirement. As  previously described, NPSTC published a report to provide guidance for  FirstNet as it constructs and implements the public safety network.  NPSTC concluded that a “public-safety grade” communications system  should be designed to resist failures due to manmade or natural events  as much as practical, and that the public safety network must be  constructed to meet as many of these requirements as possible. If  FirstNet implements all of NPSTC’s best practices, though, it will  significantly add to the cost of building the network. For example,  transmission sites, such as cell towers, should have backup power  sources when used for public safety communications, according to  NPSTC. Existing commercial sites, however, generally do not have such  backup, primarily to reduce costs in extremely competitive markets. FCC,  for instance, has reported that it could cost $35,000 per site to harden  existing commercial LTE sites.

section_title: Network Coverage
paragraphs: The public safety network’s coverage will also influence the cost to  construct and operate the network. The 2012 act requires FirstNet to  establish a “nationwide” network, but does not define the level of  coverage that constitutes “nationwide.” Generally speaking, increasing  the area covered by the network, as well as the extent to which coverage  penetrates buildings, increases the amount of infrastructure needed, and  thus the cost of the network. It may be relatively affordable, for example,  to cover large segments of the population concentrated in relatively small  areas. For instance, one stakeholder we spoke with suggested that  FirstNet could provide service to as much as 25 percent of all potential  customers by covering just the 8 largest metropolitan areas. Providing  coverage outside dense metropolitan areas can be particularly expensive.  One study has shown that a nationwide public-safety broadband network  would generally be profitable in urban areas and unprofitable in rural  areas. This study demonstrated that a network built with a commercial  partnership could cover 94 percent of the U.S. population and break even  because urban areas could subsidize coverage in rural areas.

section_title: FirstNet Faces Difficult Decisions about User Fees and Commercial Partnerships in Determining How to Become Self-Funding
paragraphs: Although FirstNet has various revenue options it is authorized to use to  become self-funding, it is unclear how FirstNet will use those authorities.  As the cost estimates discussed above illustrate, FirstNet’s network will  likely cost tens of billions of dollars to construct and initially operate. As  also noted above, FirstNet is required to be self-funding. To meet the  costs of building and maintaining the network, FirstNet may generate  revenue through user fees and commercial partnerships, the latter of  which can involve the secondary use of the network for non-public safety  services. However, FirstNet faces difficult decisions determining how to  best utilize these revenue sources. Additionally, regardless of the  effectiveness of FirstNet’s use of these revenue sources, the public safety  network will likely have net negative income in the first few years of  operation.
subsections: section_title: User Fees
paragraphs: More than 75 percent of survey respondents noted that the network would be “very  useful” to emergency management, emergency medical services, fire services, and law  enforcement. For question wording, see appendix II question 1. ability to have reliable access to broadband data throughout the state  would improve interoperability,” as well as a first responder’s ability to  deliver critical services in a timely manner.

Despite the demand for the public safety network, a variety of challenges  could hinder adoption and thus user fee revenue:

Fee size: If FirstNet’s user fee is too high it could hinder public safety  adoption, and if it is too low it could bring in too little revenue.  Numerous stakeholders we spoke with noted that FirstNet’s cost  would play a role in whether they adopt the public safety network and  that user fees must be competitive with existing commercial  services. According to a few public safety entities we spoke with and  the SPOCs we surveyed, public safety entities currently pay $20-$100  per user or device, per month, for commercial services. While low  user fees would be attractive to public safety entities and therefore  may increase adoption, they would also bring in a relatively smaller  amount of revenue per user. As a FirstNet Senior Program Manager  reported in December 2014, there is a trade-off with low user fees  between adoption and the network’s financial sustainability. Some  stakeholders also noted that the cost of equipment and devices  needed to access the public safety network could limit adoption,  especially since public safety entities are continuing to invest in their  LMR devices and equipment.

User base: While a large user base can potentially bring in significant  user fee revenue, it could be challenging to manage. The 2012 act  established that FirstNet’s primary customers will be entities that  provide “public safety services.” How FirstNet interprets the  definitions of “public safety services” established in the 2012 act will  expand or contract the potential sources of revenue. As one public  safety official we spoke with noted, the network has more value to  public safety entities when there are more users on the network,  because entities will all be able to communicate with each other.  Another public safety official we spoke with suggested that certain  users could be required to adopt the public safety network.  Government users in Sweden, for instance, are required to pay a user  fee, regardless of their use of the network, to generate revenue that is  necessary to support the national network’s maintenance and  operations. However, a large user base can require priority and  preemption rules, if certain users are to have privileged access to the  network. According to some public safety officials we spoke with, such  rules can be difficult to establish among public safety entities.

Coverage: Widespread network coverage can attract more users,  and thus user fee revenue, but is expensive to construct and maintain.  As the FirstNet Senior Program Manager reported in December 2014,  there is a trade-off with increased coverage between adoption and the  network’s financial sustainability. Further, FirstNet does not have total  control over the network’s coverage, since states may opt out and  build their own RANs. Nevertheless, some of the public safety entities  we spoke with said that the network’s coverage would play a role in  whether they adopt the public safety network, noting in particular that  the coverage should be at least as good as existing commercial  services. One public safety entity we spoke with said that existing  commercial coverage is inadequate, while two other entities said that  the coverage is adequate normally, but the service becomes unusable  during large events because of the number of users on the network.  However, as noted above, providing extensive coverage, especially in  rural areas, can be very costly. Indeed, a few SPOCs noted in survey  responses that providing rural coverage in their states would be  challenging, with one commenting that “it is inconceivable that  FirstNet will be able to deploy a terrestrial network in the vast areas  that are unpopulated or sparsely populated.”

Reliability: Although FirstNet is required to construct a resilient  network, practices to ensure this can be costly. Some public safety  officials we spoke with said that the network’s reliability would play a  role in whether they adopt the public safety network. A few officials  specifically said that if the network did not reliably work when first  utilized by public safety, adoption would suffer, since public safety has  a low tolerance for unreliable technology. However, as noted above,  ensuring reliability requires significant capital expenses. As the  FirstNet Senior Program Manager reported in December 2014, there  is also a trade-off with hardening the network between the extent of  adoption and the network’s financial sustainability.

Compounding these challenges are other factors that might hinder  adoption. For instance, officials from some public safety entities told us  that public safety tends to take a “wait and see” approach to adopting new  technology. Public safety, according to an official we spoke with, can be  reluctant to buy and use new technology and services because if those  things do not work it can put lives in jeopardy. Furthermore, according to  officials from two public safety entities we spoke with, some entities may  not see a need for a nationwide public-safety network if they operate in  areas with few large-scale emergencies.

section_title: Commercial Partnerships
paragraphs: FirstNet can also generate revenue through commercial partnerships, but  the extent of commercial interest in these partnerships, and therefore the  value of this authority for FirstNet, is currently unknown. Under the 2012  act, FirstNet can receive payment for the use of the public safety  network’s capacity by non-public safety users as well as use of the  network’s infrastructure. The value of secondary access to the public  safety network’s capacity depends in part on the availability of the  spectrum, which itself will be determined in part by the capacity available  given the network’s design. According to one major carrier we spoke with,  no business is likely to enter into a partnership with FirstNet because its  public-safety user base has not been defined, and thus the network’s  capacity available to secondary (commercial) users is unknown.  According to this carrier, the risk would be too high for a commercial  entity to enter into an agreement without knowing exactly how they will be  able to use FirstNet’s network. If public safety preempts all commercial  traffic, then the commercial entity will struggle to generate income from  this venture and may lose favor with its customers. However, another  major carrier we spoke with maintained that FirstNet will have to partner  with at least one commercial carrier to be financially sustainable, and  given the significant investments in LTE infrastructure made by  commercial carriers, FirstNet would do well to utilize some of this  infrastructure through commercial partnerships. Although the historic  Advanced Wireless Services spectrum auction that FCC concluded in late  January 2015 could indicate demand for spectrum capacity among  commercial carriers, the extent of carrier interest in partnering with  FirstNet is not yet fully known. In particular, there may be some benefits  to existing commercial carriers in partnering with FirstNet, but these  companies may prefer to expand their businesses by directly competing  with FirstNet and offering their own public safety products. Notably, when  FCC presented the D Block spectrum for auction in 2008 with public  safety encumbrances, it received no qualifying bids and thus the D Block  was not licensed. The lack of commercial interest in the D Block was due  in part to uncertainty about how the public-private partnership would work,  which raises further questions about FirstNet’s ability to partner with  commercial carriers.

section_title: Although Early Builder Projects Are Providing Lessons, FirstNet Has Not Developed a Plan to Evaluate Them
subsections: section_title: Early Builder Projects Have Learned Lessons As They Develop Public Safety Networks
paragraphs: Officials from the five early builder projects, as well as the three canceled  projects, told us that they have learned a number of lessons while  developing their public-safety broadband networks that may be useful as  FirstNet develops its public safety network. Specifically, the early builders  identified lessons about (1) governance, (2) financing the network, (3)  conducting outreach, and (4) planning for network deployment.
subsections: section_title: Governance
paragraphs: Officials from the early builder projects cited governance lessons  associated with developing a new network for public safety. As we  reported in February 2012, governance is a key element for interoperable  networks. By providing a framework for collaboration and decision  making with the goal of achieving a common objective, governance  structures can promote interoperability and help ensure public safety  networks are secure and reliable. The 2012 act established FirstNet as  the governing entity for a nationwide public-safety network, and as such,  early builder project officials described governance challenges that  FirstNet may face. For example, officials from one project told us some  public safety entities may not have a clear understanding of FirstNet’s  goals and plans. The officials told us localities are willing to participate in  the public safety network, but FirstNet will face difficulty in establishing  timely technical decisions and effective policies that keep pace with local  enthusiasm to participate. The officials said FirstNet can address this  challenge by setting expectations about what the network will provide,  including the specific intent, purpose, and planned capabilities. A SPOC  working with one of the projects described challenges that FirstNet will  face in determining how to diplomatically work with tribal nations. For  example, whereas each state will have a designated SPOC, tribes are  sovereign nations within a state but will not have a designated SPOC,  which could pose governance challenges. The SPOC told us it is unclear  whether FirstNet has planned for how the “opt in” or “opt out” process will  work for tribal nations, when the 2012 act requires state governors to  make the state decision. The SPOC also told us that a tribe  representative in his state met with him to share concerns about how  public safety entities only have broadband coverage near the edge of  their reservation’s limits. The SPOC noted that the tribe has become a  partner of the early builder project and that this partnership underscores  how their state has developed close relationships with tribes through  state-level liaison efforts, federal grant programs, and its early builder  project. According to the SPOC, FirstNet should work closely with states  and leverage these relationships to work with tribal nations.

section_title: Financing the Network
paragraphs: Officials from the early builder projects also learned lessons related to  financing a new public safety network that could be applicable to FirstNet.  Although the 2012 act provides FirstNet with funding sources and options,  as described above, utilizing such sources could be challenging  according to early builder project stakeholders. Officials from a project  told us they will face sustainability challenges due to the limited number of  users that will be able to utilize their network. According to the officials,  their project will not be able to charge their users enough to make the  operations sustainable without pricing the users out of the services. The  officials told us a possible way to address this challenge would be to  expand the service to public safety entities in neighboring metropolitan  areas and airport services areas that have established broadband  infrastructure. Similarly, officials from one of the projects said their  network will be in an “uneasy” financial position initially because the  number of users subscribing to the network will likely be low. Further, the  officials noted that a small user base could make it difficult to maintain the  network because user fees are expected to pay for a significant portion of  network operations and maintenance. According to a SPOC working with  one of the projects, determining user fees to cover ongoing maintenance  and administrative costs has been a financial challenge. To decide user  fees, project officials are considering factors including the potential for  public-private partnerships and how to define the scope of the user base.  As we found in February 2012, obtaining adequate funds to build,  operate, and maintain a public safety network could be challenging.

section_title: Conducting Outreach
paragraphs: Mobile “deployables” can also be referred to as “deployable networks.” A deployable  network typically includes “deployable assets” such as “cells on wheels” that provide  localized wireless network service to areas where coverage is minimal or compromised.  These assets typically provide fully functional service via vehicles such as trailers, vans,  and trucks. challenges in their state on the topic of network coverage. According to  the SPOC, their state’s public safety community and tribal nations  became frustrated with FirstNet’s inconsistent messaging about the  network’s coverage and capability. The inconsistent messaging created  challenges for the SPOC in convincing frustrated potential users to  remain engaged in the state’s FirstNet planning efforts. We also spoke  with a SPOC closely involved in one of the early builder projects who  described strategies that his state’s project used in conducting outreach.  For example, the project developed contact lists for each site deployment  location and for primary and secondary stakeholders, and distributed  materials in public safety locations including police stations. According to  the SPOC, FirstNet will need to sustain a level of excitement in its  outreach for the public safety network.

section_title: Planning for Network Deployment
paragraphs: Officials from the early builder projects as well as the canceled public- safety projects also described lessons they learned about planning their  network’s deployment that could benefit FirstNet. Officials from one  project told us they learned specific lessons about site selection, permits  and site access agreements, and equipment choices. The officials also  told us they selected the RFP response that was the most economical in  its use of existing infrastructure. Officials from another project told us how  they chose to deploy in densely populated areas with high-crime rates,  where public safety coordination is typically challenging and in most need  of improvement. The officials also said they provided guidance to local  police departments that helped them avoid procuring communications  devices that would not be compatible with the project or FirstNet’s public  safety network. An official from one of the canceled projects said his team  faced several challenges including local zoning conditions that affected  project schedule and cost, a newly passed city code that required towers  to withstand higher wind loads and that increased costs, and commercial  competitors lowering their subscription rates to compete with the planned  public-safety network. Officials from an ongoing project told us their  project initially identified “buildout” sites but learned that environmental  assessments would need to be completed for each site. Doing these  assessments would threaten the project’s ability to follow its project  schedule. To address the challenge, the project narrowed its buildout site  pool to exclude marshlands and other areas with obstructive tree lines  and to include publicly owned sites such as police and fire stations. With  the publicly owned sites identified, project officials worked with their  state’s legislature to pass an exemption to state environmental reviews.  According to a SPOC we spoke with, the project’s efforts on this issue will  reduce the project’s overall build time. The official from the project told us  that an important lesson learned is to thoroughly understand all of the  process steps and risks prior to plan execution. According to the project  official, it will be important for FirstNet to be able to navigate similar  issues and challenges.

section_title: FirstNet Has Processes in Place to Identify Early Builder Project Lessons but Has Not Developed a Written Evaluation Plan
paragraphs: While FirstNet has taken steps to collect and evaluate lessons learned  from the early builder projects, it could do more to ensure that the lessons  are properly evaluated. We have previously found that a well-developed  evaluation plan for projects like the early builder projects can help ensure  that agencies obtain the information necessary to make effective program  and policy decisions. A well-developed evaluation plan should include,  at a minimum, several key features including the following: well-defined, clear, and measurable objectives;  criteria or standards for determining project performance; a clear plan that details the type and source of data necessary to  evaluate the project, methods for data collection, and timing and  frequency of data collection; and a detailed data-analysis plan to track performance and evaluate the  projects’ final results. procedures and experiences. According to a SPOC involved in one of  the projects, FirstNet’s key-learning conditions cover a comprehensive set  of issues and should provide valuable data for FirstNet. Under the  SMLAs, the projects also agree to provide FirstNet with quarterly  reporting on their project’s use of FirstNet’s spectrum, progress achieving  project milestones, and in some cases, the experiences of their network  users. In October 2014, FirstNet provided the projects with quarterly  report templates, instructions, and timing for completing the reports.  Additionally, FirstNet intends to gain knowledge from the projects through  contractors who have been assigned to each project to provide  information and collect formal and informal lessons.contractors are using an informal lessons log to track observed lessons  and whether they have been incorporated into the technical documents  used to guide FirstNet’s acquisition of a comprehensive network solution.  FirstNet officials also told us that they hold weekly meetings to review  early builder project status, progress on SMLA key-learning conditions,  and informal key lessons. Finally, in April 2014 FirstNet authorized the  PSAC to establish an Early Builder Working Group to provide advice on  the strategies and lessons learned related to the early builder network  development, outreach, and consultation. As of January 2015, the PSAC  planned to submit the Early Builder Working Group’s first series of  recommendations to FirstNet.

Although FirstNet has taken these steps, we are concerned that it lacks a  detailed data-analysis plan to track the performance and results of the  early builder projects. For the early builder projects, their performance  and results are captured in the observations and lessons learned reported  to FirstNet and identified by consultants. Tracking the early builder  projects’ observations and lessons against FirstNet technical  documentation is necessary to ensure that the lessons have been  addressed and also facilitates transparency and accountability for  FirstNet’s decision-making. Even though FirstNet staff and contractors  remain in close contact with the early builder projects, without a data- analysis plan to track those projects it is unclear how FirstNet intends to  evaluate the projects’ observations and lessons and determine whether or  how the lessons are addressed. As a result, we believe that FirstNet  could miss opportunities to leverage key lessons related to governance,  finance, outreach, and network deployment. Given that the early builder  projects are doing, in part, on a regional and local level what FirstNet  must eventually do on a national level, a complete evaluation plan that  includes a detailed data-analysis plan could play a key role in FirstNet’s  strategic planning and program management, providing feedback on both  program design and execution. Furthermore, such a plan could provide  FirstNet officials the opportunity to make informed midcourse changes as  they plan for the public safety network, and help ensure that lessons from  these projects are evaluated in ways that generate reliable information to  inform future program-development decisions.

section_title: Conclusions
paragraphs: The lack of interoperability in public safety communications has been a  long-standing concern given the essential role these communications play  in protecting lives, health, and property. The 2012 act provided FirstNet  with the basic resources necessary—such as spectrum and initial  capital—to establish an interoperable broadband network for all public  safety entities. However, there is no guarantee that FirstNet will be able to  successfully develop and operate this nationwide network. FirstNet is  tasked with a complex and challenging mission to establish the network,  which researchers have estimated could cost as much as $47 billion to  construct and operate over its first 10 years. Furthermore, FirstNet faces  a multitude of risks, significant challenges, and difficult decisions in  meeting its statutory responsibilities, including determining how to  become a self-funding entity. If FirstNet fails to generate enough revenue  to operate the network over the long-term, for example, it could  jeopardize the existence of this new public safety network. At this time,  the extent to which FirstNet can generate revenue through partnerships  with commercial carriers remains unknown, especially given that some  commercial carriers could choose to compete with FirstNet. However,  FirstNet is taking certain actions to help ensure that its public safety  network is successful. For instance, to date it has effectively consulted  with stakeholders and maintained a rigorous cost estimate.

Although FirstNet has started to establish an internal control system to  help it meet its statutory responsibilities, we found FirstNet could  strengthen its internal controls. In particular, while FirstNet has begun  taking some steps to assess risks, it has not fully assessed the risks it  faces. Complete risk assessments would help FirstNet respond to risks in  a proactive manner and make the best use of its resources by  appropriately responding to the most pressing risks. FirstNet could also  strengthen its internal controls by fully establishing its control  environment, which is the foundation for an effective internal control  system. Currently, FirstNet has not established a cohesive standards of  conduct policy, which means FirstNet may not be able to address  deviations in conduct and performance and take corrective actions in a  timely manner. Establishing and evaluating adherence to standards of  conduct would help FirstNet ensure that all its personnel are held  accountable for their actions and foster stakeholder trust in FirstNet’s  ability to meet its statutory responsibilities.

Early builder projects have learned important lessons related to  governance, finance, outreach, and network deployment that could be  useful to FirstNet as it develops its plans to establish a nationwide  network. However, FirstNet lacks a detailed data-analysis plan to track  the projects’ observations and lessons learned. Without such a plan, it is  unclear how FirstNet intends to evaluate the early builder projects and  ensure that the lessons have been addressed and incorporated, if  applicable, into FirstNet’s planning. As a result, FirstNet could miss  opportunities to leverage the key lessons the projects learned. A  complete evaluation plan for the early builder projects that includes a  detailed data-analysis plan would increase transparency and help  FirstNet’s strategic planning and program management, which are  important given the complexity of FirstNet’s mission. Furthermore, such a  plan would provide FirstNet officials the opportunity to make informed  midcourse changes as they plan the nationwide network and help ensure  that lessons from these projects are evaluated in ways that generate  reliable information to inform future network-deployment decisions.

section_title: Recommendations for Executive Action
paragraphs: To improve the accountability and transparency of FirstNet’s operations,  and ensure that FirstNet is gaining as much knowledge from the early  builder projects as possible, we recommend that FirstNet take the  following two actions:  strengthen FirstNet’s internal control system by fully assessing risks,  developing standards of conduct, and evaluating performance against  these standards, and develop an evaluation plan that includes a detailed data-analysis plan  for the early builder projects’ performance and results, including how  the observations and lessons learned reported to FirstNet and  identified by consultants will be evaluated.

section_title: Agency Comments
paragraphs: We provided a draft of this report to the Departments of Commerce and  Homeland Security, FCC, and FirstNet for their review and comment. The  Department of Commerce and FirstNet provided written comments,  reprinted in app. III and IV, respectively. DHS, FCC, FirstNet, and NTIA  (within the Department of Commerce), provided technical comments that  we incorporated as appropriate.

In its written comments, the Department of Commerce stressed that it  takes its oversight responsibilities with respect to FirstNet seriously, is  committed to the success of the public-safety broadband network, and  supports the response provided to us from FirstNet. In its written  comments, FirstNet stated that it agreed with all of our recommendations  and noted activities that it will undertake to implement them. Regarding  strengthening internal controls, FirstNet stated that it is cognizant that, as  a newly formed government entity, it must continue its efforts to establish  comprehensive internal control policies and procedures. FirstNet  reiterated that it has initiated a legal compliance risk assessment focusing  on key legal risk areas and stated that it will also undertake a full risk  assessment. FirstNet also stated that it plans to establish supplemental  standards of conduct, which will operate in conjunction with applicable  regulations and existing FirstNet and Department of Commerce policies.  Regarding an evaluation plan for lessons learned from the early builder  projects, FirstNet stated that it will develop an appropriate evaluation plan  consistent with the principles we specified. FirstNet noted that it has  already enacted a standard operating procedure that ensures information  and lessons from the projects are appropriately distributed within FirstNet,  and it will use this process to disseminate the findings gathered under the  evaluation plan.

We will send copies of this report to FirstNet as well as the Secretary of  Commerce, Secretary of Homeland Security, Chairman of the Federal  Communications Commission, and appropriate congressional  committees. In addition, the report will be available at no charge on  GAO’s website at http://www.gao.gov.

If you or members of your staff have any questions about this report,  please contact me at (202) 512-2834 or goldsteinm@gao.gov. Contact  points for our Offices of Congressional Relations and Public Affairs may  be found on the last page of this report. Major contributors to this report  are listed in appendix V.

section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: This report examines the First Responder Network Authority (FirstNet)  and its progress towards establishing a nationwide public-safety  broadband network (hereafter, the public safety network). Specifically, we  reviewed (1) the extent to which FirstNet is carrying out its responsibilities  and establishing internal controls for developing the public safety network,  (2) how much the public safety network is estimated to cost to construct  and operate and how FirstNet plans to become a self-funding entity, and  (3) what lessons can be learned from local and regional public-safety  network projects.

To determine the extent to which FirstNet is carrying out its  responsibilities and establishing internal controls, we reviewed FirstNet  documentation and conducted interviews. We reviewed all of FirstNet’s  Requests for Information, Notices, and annual reports to Congress. We  also reviewed relevant board-meeting materials and resolutions,  presentations to stakeholders, press releases and blog posts, and other  documentation. We interviewed and received written responses from  FirstNet, the National Telecommunications and Information  Administration, and Department of Commerce officials to obtain further  information on their efforts. We compared FirstNet’s efforts and progress  carrying out its responsibilities against requirements established in the  Middle Class Tax Relief and Job Creation Act of 2012 (2012 act).  However, we did not review FirstNet’s progress against every  responsibility established for it in the 2012 act, because it is not possible  for FirstNet to have made progress on some responsibilities. For  instance, FirstNet cannot develop terms of service for the use of the  public safety network without first developing the network, the design of  which is itself dependent on consultation with a wide variety of  stakeholders. We did compare FirstNet’s efforts establishing internal  controls against criteria established in the federal Standards for Internal  Control.policies and practices against the first two components of internal control:  control environment and risk assessment. We chose these two  components because the control environment is the foundation for an   For the scope of this review, we chose to evaluate FirstNet’s  internal control system and risk assessment provides the basis for  developing appropriate risk responses and control activities. We also  assessed FirstNet’s outreach efforts against core principles for effective  stakeholder participation identified by our previous reports. Further, as  described below, we interviewed a variety of public safety officials about  their perspectives on FirstNet’s progress to date.

GAO, GAO Cost Estimating and Assessment Guide: Best Practices for Developing and  Managing Capital Program Costs, GAO-09-3SP (Washington, D.C.: Mar. 2, 2009). detailed understanding of the cost model, such as how it was prepared,  the assumptions underlying it, and the documentation supporting it. We  also interviewed FirstNet officials about how it plans to become self- funding.

To assess the factors that will influence the cost of the public safety  network and challenges FirstNet may face in becoming self-funding, we  conducted a variety of interviews and reviewed documents. As described  below, we interviewed officials involved in early builder projects, as well  as state and local public-safety entities. We also interviewed subject  matter experts who were interviewed for our previous report on  emergency communications. Additionally, we interviewed two major  commercial wireless carriers for their perspectives on building and  operating a public safety network. We also reviewed FirstNet  documentation, academic literature suggested to us by subject matter  experts, and reports published by FCC and the National Public Safety  Telecommunications Council (NPSTC).

To identify lessons that can be learned from local and regional public- safety broadband network early builder projects, we interviewed project  officials and reviewed documentation from FirstNet and the projects.  Specifically, we conducted site visits and phone interviews with officials  from—and involved in—the five current projects (Los Angeles, CA;  Adams County, CO; New Jersey; New Mexico; and Harris County, TX),  the three projects that were canceled (Charlotte, NC; Mississippi; and  San Francisco, CA), and state and local public-safety entities in the  project jurisdictions. We also reviewed the Spectrum Manager Lease  Agreements that each of the five current projects established with  FirstNet, and documentation related to how FirstNet plans to collect  lessons learned from the projects, such as Key Learning Condition Plans,  project quarterly reporting requirements, and other documentation  provided by project officials. We also interviewed and obtained written  responses from FirstNet officials to obtain more information about their  plans to evaluate and utilize lessons from the projects. We assessed  FirstNet’s plans to evaluate and utilize lessons from the projects against  key features of a well-developed evaluation plan for pilot projects  identified by our previous reports.

The 5 U.S. territories we surveyed were American Samoa, Guam, the Northern Mariana  Islands, Puerto Rico, and the U.S. Virgin Islands. service, three emergency management or communications, and two  general public safety entities.

section_title: Appendix II: Copy of Survey to FirstNet Single Points of Contact

section_title: Appendix III: Comments from the U.S. Department of Commerce

section_title: Appendix IV: Comments from the First Responder Network Authority

section_title: Appendix V: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact

section_title: Staff Acknowledgments
paragraphs: In addition to the individual named above, Sally Moino (Assistant  Director), Susan Baker, Kyle Browning, David Hooper, Kristen Kociolek,  Abishek Krupanand, Jason Lee, Josh Ormond, Nalylee Padilla, Amy  Rosewarne, Kelly Rubin, Grant Simmons, Andrew Stavisky, and Michael  Sweet made key contributions to this report.


Question-summary annotations: questions: question: What are FirstNet's responsibilities?
answer: FirstNet is charged with the complex and challenging task of establishing a new, nationwide, wireless broadband network for public safety entities, in consultation with federal, state, local, and tribal stakeholders.
child_questions: question: What will be included in the wireless network?
answer: The network will initially support interoperable data communications, and later integrate mission-critical voice capabilities as public safety standards for voice communications are developed.

question: To what extent has FirstNet accomplished its goals?
answer: The First Responder Network Authority (FirstNet) has made progress carrying out its responsibilities established in the Middle Class Tax Relief and Job Creation Act of 2012 (the 2012 act) but lacks certain elements of effective internal controls.
child_questions: question: What progress has FirstNet made?
answer: FirstNet has made progress establishing an organizational structure, planning for the network, and consulting with stakeholders. FirstNet has also begun establishing policies and practices consistent with federal internal control standards. Officials told GAO that they plan to continue to do so.

question: What are the gaps in this progress?
answer: However, FirstNet has not fully assessed its risks or established standards of conduct—which is an important form of ethical guidance for its personnel .
child_questions: question: Why is assessment important for FirstNet?
answer: Given that FirstNet faces numerous risks to achieve its complex objectives, fully assessing risks could help FirstNet achieve its objectives and maximize use of its resources.

questions: question: What difficult decision is FirstNet facing?
answer: A nationwide public-safety broadband network has been estimated by various entities to cost billions of dollars, and FirstNet faces difficult decisions determining how to fund the network's construction and ongoing operations.
child_questions: question: What is the estimated scale of the cost?
answer: These estimates indicate the cost to construct and operate such a network could be from $12 to $47 billion over the first 10 years.

question: How might the actual cost be influenced?
answer: The actual cost of FirstNet's network will be influenced by FirstNet's (1) business model, especially the extent of commercial partnerships; (2) use of existing infrastructure; (3) efforts to ensure network reliability; and (4) network coverage. For example, the cost of the network may be higher if FirstNet does not utilize partnerships and some existing infrastructure.

question: What is needed for FirstNet to become self-funding?
answer: To become self-funding, FirstNet is authorized to generate revenue through user fees and commercial partnerships. However, FirstNet faces difficult decisions in determining how to best utilize these revenue sources.

questions: question: What is the current status of FirstNet's evaluation efforts?
answer: FirstNet has taken steps to collect and evaluate information and lessons from the five “early builder projects” that are developing local and regional public-safety networks, but could do more to ensure that the lessons are properly evaluated.
child_questions: question: What is one example of this current status?
answer: For example, FirstNet has asked the projects to report on the experiences of their networks' users and has assigned contractors to collect and log lessons. However, FirstNet does not have a plan that clearly articulates how it will evaluate those experiences and lessons.
child_questions: question: What has GAO determined about such projects?
answer: Although FirstNet told GAO that it remains in close contact with early builder projects, GAO has previously found that a well-developed evaluation plan for projects like these can help ensure that agencies obtain the information necessary to make effective program and policy decisions.

question: Why would a better evaluation plan help FirstNet?
answer: Given that the early builder projects are doing on a local and regional level what FirstNet must eventually do nationally, an evaluation plan can play a key role in FirstNet's strategic planning and program management, providing feedback on both program design and execution and ensuring FirstNet has not missed opportunities to incorporate lessons the projects have identified.

questions: question: On what do public safety officials rely for emergency communications?
answer: For communications during emergencies, public safety officials rely on thousands of separate systems, which often lack interoperability, or the ability to communicate across agencies and jurisdictions.

question: Why was FirstNet created?
answer: The 2012 act created FirstNet within the Department of Commerce to establish, for public safety use, a nationwide, interoperable, wireless broadband network, which will initially support data transmissions.
child_questions: question: What specifics did the 2012 act lay out?
answer: The 2012 act established numerous responsibilities for FirstNet, provided $7 billion for network construction, and required FirstNet to be self-funding beyond this initial allocation.

question: How has FirstNet begun this work?
answer: As part of the effort, FirstNet is working with five “early builder projects” that are building local and regional public-safety broadband networks.

questions: question: How did the GAO examine FirstNet's progress in establishing the network?
answer: GAO was asked to examine FirstNet's progress in establishing the network. GAO assessed (1) FirstNet's progress carrying out its responsibilities and establishing internal controls, (2) how much the network is estimated to cost and how FirstNet plans to become self-funding, and (3) what lessons can be learned from the early builder projects.
child_questions: question: How did GAO conduct this evaluation?
answer: GAO reviewed FirstNet documentation and public-safety network cost estimates, surveyed all state-designated FirstNet contacts, and interviewed FirstNet officials and public safety stakeholders selected for their telecommunications and public safety experience.