Recovery Act: Broadband Programs Are Ongoing, and Agencies' Efforts Would Benefit from Improved Data Quality
Released: 2012-09-14T00:00:00

Letter

Background

In addition to being administered by different agencies, BTOP and BIP differ in several ways, including in methods for providing funding to awardees, types of awardees funded, and services funded through the programs. NTIA funded a range of organizations from states and municipalities to non-profit and commercial organizations, whereas RUS made BIP loans, grants, and combinations of loans and grants primarily to private-sector entities, including for-profit companies and cooperatives. In addition to funding infrastructure projects, BTOP funded public computer center (PCC) projects and sustainable broadband adoption (SBA) projects. Also, the vast majority of BTOP’s infrastructure awards were investments in “middle-mile” infrastructure projects, which provide a link from the Internet backbone to the last mile networks of local providers (such as cable or phone companies) that provide broadband service to end users. The availability of broadband service to end users depends upon access to adequate middle-mile facilities, which can be costly to deploy in rural areas. In contrast, BIP primarily funded “last-mile” infrastructure projects that provide service directly to end users. Table 1 provides more detail about the BTOP and BIP programs, including the current number of projects and the amount of funding they represent.

NTIA and RUS solicited applications to the programs and made awards in two rounds, with the first funding round beginning in June 2009 and ending in April 2010, and the second round beginning in January 2010 and ending in September 2010. In the joint Notice of Funds Availability, NTIA and RUS provided that projects should be substantially complete within 2 years of receiving an award. The agencies disburse awarded funds as projects progress. To meet the substantially complete requirement, a project must receive two-thirds of its award 2 years after receiving the award. NTIA and RUS also provided that projects should be complete within 3 years of receiving an award. In October 2011, RUS modified these requirements to provide that BIP projects must commence within 180 days of the latter of the completion of the project’s historic preservation or environmental review, and be fully complete no later than June 30, 2015. NTIA maintained the original requirements. Since NTIA made several rounds of awards, the 3-year completion deadlines for the projects are staggered throughout 2012 and 2013. Table 2 shows the various deadlines, and the number of BTOP projects subject to that deadline.

NTIA and RUS must oversee BTOP and BIP projects, respectively, through each program’s completion. NTIA and RUS both have officials monitoring the overall progress of the programs and reviewing requests for funds. In addition to these officials, the agencies also provided staff to serve as the primary point-of-contact for their awardees and answer questions and address issues that arise for specific awardees. NTIA’s Federal Program Officers (FPO) provide this assistance to BTOP awardees; RUS’s General Field Representatives (GFR) serve the same role for BIP awardees. In addition, BTOP and BIP awardees face a variety of reporting requirements. For example, the Recovery Act requires fund recipients, including BTOP and BIP awardees, to submit quarterly reports that provide a description of their projects or activities, the progress of their projects, and estimates of the number of resulting jobs funded, measured on a FTE basis.awardees submit quarterly and annual progress reports to the agency that provide financial data and information on the projects’ status and effects. RUS requires that BIP awardees submit quarterly reports that provide subscribership data, and according to RUS, these efforts also include financial statements consisting of income statements, balance sheets, and cash flow statements.

The Progress of Recovery Act Broadband Projects Is Difficult to Measure because of Data Limitations

NTIA and RUS both use the amount of funds disbursed to awardees as one method of tracking progress, and less than half of all the awarded funding has been disbursed to Recovery Act BTOP and BIP awardees. BTOP awardees appear to have made more progress than BIP awardees, as NTIA has disbursed half of BTOP’s funds while RUS has disbursed less than one-third of BIP’s funds. However, the agencies disburse awarded funds for projects as payment becomes due, sometimes only as contracts are complete. Because disbursements do not fully reflect the amount of work completed, NTIA has established other non-financial indicators of progress, such as network miles deployed. In contrast, RUS did not initially establish indicators to measure the deployment of infrastructure.

Amount of Funding Disbursed as an Indicator of Progress

As awardees implement their projects, they request funds from the agency administering their award. NTIA and RUS track how much of each project’s award that they have disbursed to the awardee, which serves as one indicator of progress. The data that NTIA and RUS collect indicate that the agencies have disbursed less than half of all awarded funds. Of the roughly $3.8 billion available for the BTOP projects, NTIA has disbursed approximately $1.9 billion (50 percent) to its awardees. As noted in figure 1, disbursements to NTIA’s infrastructure projects comprise $1.6 billion of BTOP’s disbursed funds, while disbursements to public computer center and sustainable broadband adoption projects total $290 million. Of the roughly $3.3 billion available for the BIP projects, RUS has disbursed approximately $1 billion (30 percent) to its awardees.

Our analysis of the disbursements indicates that BTOP awardees have made more progress than BIP awardees. Sixty-eight percent of BTOP awardees have received more than half of their awarded funds, versus 24 percent of BIP awardees. The fact that the BTOP program includes non- infrastructure projects, such as public computer center and sustainable broadband adoption projects, may partially explain this disparity; NTIA officials noted that since most of these non-infrastructure projects are not required to undergo an environmental assessment, they tend to progress more quickly than the infrastructure projects. See table 3.

Of the BTOP projects that have reached the 2-year target to be substantially complete, more than half have received two-thirds of their total award. As previously mentioned, NTIA provided that BTOP projects should receive two-thirds of their total award within 2 years of receiving the award. There are 102 projects that started before August 2010 and thus have reached the two-thirds benchmark in terms of project time elapsed. Of these 102 projects, 61 have received two-thirds of their awarded funds as of July 2012. NTIA officials told us that in cases where a project does not meet the two-thirds complete threshold, NTIA program officers follow up to determine whether the project is delayed, or whether disbursements are not reflecting the amount of work completed, as mentioned above. In these cases, program officers will conduct a hands- on review, which helps NTIA determine whether the project needs a performance improvement plan with specific actionable recommendations to get the project back on track.

While disbursements are one measure of progress, officials from NTIA and RUS told us that disbursements do not fully reflect the amount of work completed. NTIA and RUS disburse awarded funds for projects as payment becomes due, sometimes only as contracts are completed. Thus disbursements do not fully reflect the amount of work completed at any point in time. NTIA officials noted that in some cases, awardees have entered into contracts that are structured so that the awardee does not pay the contractor until the bulk of construction is complete. For example, one awardee we interviewed noted that although the project was 95 percent complete, it had only received 65 percent of the project’s funding, giving the incorrect impression that the project was behind schedule. In our review of BTOP awardees’ first quarter 2012 reports to NTIA, we found that some recipients cited this issue when reporting their financial progress in the quarterly reports. Others noted that cost savings made their disbursements appear as if the project were behind schedule, but that was not the case. In other words, a project that is on schedule but incurring costs below the levels anticipated in its original plan could appear to be behind schedule, since its disbursements will be lower than expected.

Non-Financial Indicators of Progress

NTIA established other non-financial performance measures that show that awardees have made progress in implementing their projects. Specifically, NTIA established performance metrics for measuring BTOP’s progress against key performance indicators, such as the number of network miles deployed and leased, community anchor institutions connected, new workstations in public computer centers, and new broadband subscribers. NTIA established the baselines for these key performance indicators by aggregating the information awardees provided in their applications to the program, and provided program targets to the Office of Management and Budget (OMB). NTIA tracks progress against these indicators and reports that as many as 76 percent of network miles are complete and more than 97 percent of new workstations are complete. These results, when compared with the amount of funds disbursed, are consistent with NTIA’s statements that in some cases, disbursement of funds lags behind progress of projects.

RUS is collecting data on subscribers to show the effect of BIP on broadband adoption, which we discuss later in this report. However, we do not consider this a measure by which one can evaluate the progress of a project toward completion. disbursed, the project is built, and the agency is repaid, instead of tracking project outcome information. In addition, the Recovery Act did not require RUS to collect performance metrics from awardees. Nevertheless, the Recovery Act sought to provide an unprecedented level of transparency with respect to how funds were being spent and program outcomes were being achieved, and we have noted the importance of collecting complete, accurate, and consistent data to document program performance. In June 2012, RUS officials told us that they began tracking the number of fiber miles and wireless access points deployed by BIP projects; however, they could not ensure the quality of the data at that time. Without reliable information on the progress of BIP projects in expanding infrastructure and moving toward completion of projects, RUS may not be able to demonstrate the progress and effectiveness of the BIP program.

NTIA Has Expanded Access to Broadband through BTOP Projects; However, Data Limitations Make it Difficult to Measure the Effects of BTOP and BIP on Broadband Adoption

NTIA collects data on network miles deployed, community anchor institutions connected, and workstations added at public computer centers, which helps illustrate that BTOP expanded broadband infrastructure and provided increased access to the public. As previously mentioned, RUS did not begin collecting this type of information until this year and cannot assure the quality of the data; thus, BIP’s contribution to these goals is less clear. Both NTIA and RUS face challenges in ensuring the quality of subscribership data, and therefore, it is difficult to measure the effect of BTOP and BIP on broadband adoption.

BTOP Expanded Infrastructure and Provided Access at Public Computer Centers

As previously mentioned, the current goal of BTOP’s Comprehensive Community Infrastructure (CCI) projects is to deploy or upgrade 75,000 miles of broadband infrastructure and provide broadband access to 15,000 community anchor institutions, such as schools, libraries, health care providers, and public safety entities. As of March 31, 2012, BTOP’s 117 infrastructure projects reported that they have established over 57,000 new or upgraded network miles, with connections to over 8,000 community anchor institutions. See table 4.

As shown in table 4, BTOP projects report several different types of miles of broadband infrastructure: “Network miles installed” includes newly constructed miles, such as new aerial or buried fiber, built using BTOP funds. “Network miles leased” includes new network capacity acquired via lease using BTOP funds. “Existing network miles upgraded” refers to existing capacity that was upgraded using BTOP funds. “Existing network miles leased” refers to capacity that was acquired from a third-party prior to BTOP and that is being used or improved using BTOP funds.

In our discussions with BTOP awardees, they explained how their individual projects expanded broadband access in their communities. For example, one awardee who received BTOP and BIP funds explained that the majority of the areas served by its projects did not have broadband service, with one community lacking basic telephone service. The awardee added that as broadband expands in the area, the local economy has benefited as tourism-oriented businesses are better able to provide Web sites and online reservation systems. Another awardee noted that its project serves some areas that previously had broadband service; however, the choice of service provider was limited, and the broadband options were generally expensive and slow. This awardee reported that its BTOP-funded infrastructure improved broadband speed for community anchor institutions, including schools, community colleges, and health care providers, from 1 gigabyte per second to 10 gigabytes per second.

Interviewees also reported that the public computer center (PCC) program expanded access to broadband in their communities. BTOP’s 65 PCC projects have deployed new workstations (computers) at multiple locations, such as community colleges and public libraries, throughout a city or state. For example, the New Jersey State Library reported that its PCC project has deployed 845 computers to 124 public libraries, and 128 laptops that community colleges use to provide workforce development courses to the public. Some public computer centers also received upgraded broadband connections and wireless routers. Officials from the State Library of Louisiana noted that residents will use wireless hot spots in the library to access the Internet from their own devices, adding that one woman brings her children to the library parking lot after hours so that they can access the signal and work on their homework when the library is closed. In addition, PCC projects have installed workstations designed to be accessible by users with visual impairments. The workstation in figure 2 includes software that reads aloud what is on the computer screen, screen magnification software, and a desktop magnifier that enlarges materials placed under it. As of March 2012, PCC awardees reported that they provided nearly 34,000 new workstations for public use.

PCC projects have also provided classes on a variety of topics, including English as a second language, digital literacy, and job training. Officials from the New Jersey State Library project noted that while they initially targeted people with limited or no computer skills and provided basic computer training, they encountered unexpectedly high demand and expanded course offerings to reach recent college graduates, those with managerial experience, as well as offering additional resources for small businesses and entrepreneurs. We visited classes in Louisiana and Washington, D.C., and observed participants learning software programs. The State Library of Louisiana noted in its first quarter 2012 report that its various training programs had over 19,000 participants. During our visit, we observed a training class, and participants stated that they had driven several hours to attend some classes, and wanted to attend more. For more information see, the sidebar titled “Example of PCC Project: State Library of Louisiana.”

Measurement and Data Limitations Make It Difficult to Assess the Programs’ Effects on Broadband Adoption

NTIA and RUS have faced difficulties in ensuring that awardees provide reliable data regarding broadband subscribership for their BTOP and BIP projects, which makes it challenging to fully and accurately determine the effects of the programs on broadband adoption. NTIA’s sustainable broadband adoption (SBA) projects were established to stimulate demand for broadband Internet access, and applicants were required to describe how they would calculate subscribership for their projects. For example, the SBA project Computers for Youth/Los Angeles Unified School District Broadband Engagement Program provides 4-hour workshops in high- poverty Los Angeles schools, during which it provides hands-on, bilingual training to sixth-grade students and their families on using broadband for educational purposes. The participants receive a free computer pre- loaded with educational software as well as access to toll-free bilingual help-desk support, and Computers for Youth conducts a survey before and after the workshop to determine whether the families subscribed to broadband service after attending the workshop. Computers for Youth stated in its first quarter 2012 report that over 30,000 participants (students and their parents) had attended a family-learning workshop during the course of its program and that nearly 5,000 households had newly subscribed to broadband. Program officials noted that by highlighting the relevance of broadband to children’s education and offering free computers and technical support, they were able to generate broadband subscribers despite the lack of discounted broadband service.

Although current data reflect an increase in subscribers as a result of BTOP projects, they may not be accurate. As of March 31, 2012, SBA projects reported that they generated over 334,000 broadband subscribers. However, NTIA officials told us that many recipients have faced difficulties in measuring broadband subscribers related to the project and have revised their counting methods. Thus, NTIA officials noted that the total subscribers reported to date may not represent the true number of new subscribers. In our review of awardees’ first quarter 2012 reports, we found that 13 of 43 SBA projects reported difficulties with collecting subscriber data. For example, some awardees reported that they thought they would be able to obtain subscribership data from Internet service providers to determine the effect of their projects, but later found that Internet service providers were unwilling to provide subscribership data. NTIA has provided training on this issue, and facilitated sessions in which projects share best practices for measuring subscribership. Program officials told us NTIA is also working with individual projects to help them find ways to address the challenges involved in collecting subscribership data.

Similarly, RUS data on the effects of BIP projects on broadband adoption may not be accurate. RUS established a goal for BIP projects to provide new or improved broadband service to 359,450 subscribers, and in 2010, RUS stated that it exceeded this goal because it estimates that BIP projects will provide new or improved service to 847,239 subscribers. However, this total does not reflect actual program outcomes, because it is based on the estimates of applicants prior to the execution of their funded projects. RUS requires recipients to report quarterly on the number of households, businesses, educational providers, libraries, health care providers, and public safety providers receiving new or improved broadband service. However, when we reviewed the data, we noted discrepancies. When we asked RUS about these discrepancies in the subscribership data, RUS officials noted that the data are inaccurate and that RUS has implemented quality checks to improve the information. Specifically, RUS developed a spreadsheet tool that RUS staff use to review the subscriber data submitted by individual projects, note whether the data appear to be correct, and report how any problems with the data were resolved. In its comments on a draft version of this report, RUS noted that its field staff provides extensive contact and guidance to awardees.

NTIA and RUS Have Acted to Address the Variety of Challenges Awardees Identified in Completing Projects

BTOP and BIP awardees identified multiple challenges in completing projects, including compliance with regulations and construction related challenges. Additionally, BIP awardees identified some additional challenges related to RUS’s processes and requirements, and non- infrastructure projects participating in BTOP face challenges related to staffing and deploying the projects. In some cases, these challenges have contributed to the lack of progress discussed above. NTIA and RUS have taken a number of actions to help awardees address these challenges, including providing awardees with regular contact, expertise, webinars, and guidance.

Challenges

While BTOP and BIP awardees have identified some of the same challenges they have faced, they also identified some different challenges because the programs are administered by different agencies and fund different services. Table 5 lists the challenges we identified facing BTOP and BIP awardees based on our interviews and content analysis of NTIA and RUS documents.

Pub. L. No. 91-190, 83 Stat. 852 (1970), as amended, codified at 42 U.S.C. ch. 55. assessments because of the large volume of Recovery Act work. The initial environmental assessment process was also the most common challenge cited by RUS’s GFRs, who told us that these assessments took longer than anticipated and negatively affected all awardees’ schedules. Officials from NTIA and RUS both noted delays associated with the environmental assessment process, and NTIA officials noted that these delays have been up to 6 months for BTOP projects.

BTOP and BIP awardees reported facing other delays in beginning project construction, such as:

Prevailing wage requirements. The Recovery Act requires that all laborers and mechanics employed by contractors and subcontractors on projects funded directly by or assisted in whole or in part by and through the federal government under the Recovery Act be paid at prevailing wage rates. NTIA and RUS officials and an awardee told us that in many cases identifying the prevailing wage for activities such as fiber deployment was difficult because that type of work had often not been done in that area and the prevailing wage had not yet been determined by the Department of Labor.

Permitting, right-of-way agreements, and make-ready work. Awardees must apply for and receive permits and right of way agreements before beginning construction. NTIA officials and awardees told us that for some projects, obtaining these agreements from all the relevant stakeholders took longer than anticipated and put the projects behind schedule. For example, these agreements may have to be obtained from state and local governments, tribal governments, federal entities (such as the Bureau of Land Management), private landowners, and railroads. One awardee we spoke with experienced delays of up to 9 months in the installation of equipment and fiber because of the amount of time it took to receive right of way permits from railroad companies. In addition, make-ready work, which includes the tasks associated with preparing utility poles for the installation of equipment and fiber, must be complete before any construction occurs, and this need has posed challenges to maintaining project timelines. For example, one BTOP awardee indicated in its first quarter 2012 report to NTIA that “keeping make-ready completion ahead of construction crew availability remains the major challenge for the project.”

Weather, terrain, and broadband fiber availability. Weather-related challenges and difficult terrain can cause delays in construction. Awardees reported delays for, among other things, a major tornado that caused damage to a service area and equipment, hurricanes, snowfall, and flooding. Difficult terrain can also cause delays. One awardee told us that its project unexpectedly called for boring through solid granite under a river, which required additional tools and time. NTIA and RUS officials told us that BTOP and BIP projects were also delayed due to fiber shortages caused by the 2011 tsunami in Japan and increased worldwide demand for fiber. For example, one BTOP infrastructure awardee noted in its first quarter 2012 report to NTIA that it was “experiencing delays in the delivery of the fiber… while we received a few shipments during this quarter it was much less than promised.”

RUS officials also noted some challenges specific to the BIP program, such as:

Approval of contracts and plans. RUS officials told us that to ensure quality construction is completed, they require that awardees gain approval of contracts before the construction begins or materials are purchased. RUS officials said that on average, it takes the agency 1 and a half months to 2 months to approve a contract. However, some RUS GFRs and awardees said that RUS did not approve BIP contracts in a timely fashion in some instances, which resulted in project delays. In particular, some GFRs said that RUS took 6 to 12 months to approve a contract. RUS officials noted that there could be a number of reasons why contracts had not been approved, including that the contracts were not correct when they were submitted for approval.

Compliance with reporting requirements. RUS officials told us some awardees that received a loan or loan/grant combination have experienced challenges complying with RUS reporting requirements. The officials explained that awardees who had not previously borrowed from RUS are more likely to experience difficulties complying with reporting requirements than those awardees with a history of borrowing from RUS.

Proposed changes to the Federal Communications Commission’s (FCC) Universal Service Fund. Some RUS officials were concerned about how reforms to FCC’s Universal Service Fund, which provides telecommunications funding to some BIP awardees, could potentially affect projects’ income streams. FCC’s reforms include changes to the distribution and use of Universal Service Fund payments to make the funds available to support both telephone service and broadband deployment. As a result of the reforms, rural telecommunications carriers could receive reduced universal service support payments. In August 2011, RUS provided data to FCC on how these changes could potentially affect the income stream of RUS borrowers and grant awardees. Subsequently, in its 2011 Order and Further Notice of Proposed Rulemaking on this issue, FCC stated that the reforms it adopted were more modest than the estimates used by RUS to determine the potential impact of the reforms, and that RUS did not consider the potential for borrowers to adopt operational efficiencies that would offset reductions in universal service support.

NTIA officials and awardees also identified challenges that apply to non- infrastructure BTOP projects, such as:

Staffing or expertise. PCC and SBA projects faced staffing and expertise challenges, such as high staff turnover, or the need for additional staff to handle tasks such as providing technical support for computers or manning computer labs. Our analysis of BTOP awardees’ first quarter 2012 reports to NTIA indicate that 7 of 43 SBA projects and 22 of 64 PCC projects reported challenges with either a lack of staff or lack of staff expertise.

Contracting, procurement, and financing. Contracting and procurement issues also commonly caused delays at the beginning of PCC and SBA projects, according to NTIA officials. These projects can be run by state or local entities, and four of the Federal Program Officers (FPOs) we spoke with told us that the projects they oversee were delayed by the need to comply with state or local contracting or procurement requirements. For example, FPOs told us that one project was delayed by 4 to 5 months because the state legislature needed to approve the spending of BTOP funding, and another project run by a large city was delayed by more than a year because of its internal contracting process.

Technical issues. Issues related to equipment, software, and Internet service, have also been challenging to SBA and PCC projects. Our analysis of BTOP awardees’ first quarter 2012 reports to NTIA indicate that 8 of 43 SBA projects and 16 of 64 PCC projects noted challenges with the technical aspects of the projects. For example, one awardee reported that software conflicts caused system crashes for the workstations deployed for seniors and adults with disabilities.

Outreach and program participation. Some awardees reported outreach and participation challenges, such as difficulty in participant retention. For example, one awardee reported that most of the participants were senior citizens and were easily discouraged when they felt that they could not retain new skills after each class or training session. Overall, according to our analysis of BTOP awardees’ first quarter 2012 reports to NTIA, 13 of 43 SBA projects and 10 of 64 PCC projects reported challenges related to outreach or participation.

Agencies’ Actions to Address Challenges

NTIA and RUS provided resources and took action to help awardees address their challenges. Both agencies have provided awardees with regular contact and support. For example, NTIA officials told us that they have regularly scheduled calls between awardees and FPOs to encourage dialog and early resolution of problems. The officials also noted they have encouraged awardees to come to the agency for help, especially in complex situations where federal influence can help resolve challenges. For example, NTIA officials stated that they worked with FCC and state public utility commissions to ensure that they prioritized their review of BTOP projects’ requests for permits. Likewise, RUS officials told us that GFRs and other agency staff have provided regularly scheduled contact to determine project status and challenges early on so they can begin addressing them. Both FPOs and GFRs told us that they worked to identify options for dealing with fiber shortages and provided that information to awardees.

Both NTIA and RUS developed webinars and training sessions to inform awardees and provide them with updated information. NTIA has provided guidance to awardees through webinars and conference calls and has held several webinars on topics including pole attachment issues, computer recycling and refurbishing, and numerous other topics. Some awardees told us the webinars and conference calls were both useful and timely. RUS also developed and held webinars focused on assisting grant and loan recipients with the contract process; which according to officials, could increase the likelihood that the contracts are approved more expeditiously so that awardees can be reimbursed in a more timely fashion. NTIA also has a monitoring process to identify projects with schedule, performance, or other challenges, and as needed provides such projects with performance improvement plans and corrective action plans. In its comments on a draft version of this report, RUS noted that it has implemented a detailed review process to identify issues with BIP awards and is constantly providing guidance and assistance to the awardees to overcome issues with the performance of the award.

In addition to providing the general resources and actions described above, NTIA and RUS took the following actions to address some of the specific challenges described earlier in this report:

NTIA took steps to facilitate the sharing of information and best practices among projects by identifying projects that had already successfully addressed a particular issue and asking them to share their experiences with similar projects. For example, a significant number of infrastructure projects were experiencing similar make-ready challenges, and NTIA organized and moderated a discussion where awardees discussed problems and solutions.

NTIA also created different groups for PCC and SBA projects to share information on a range of issues that were challenging awardees related to K-12 education, seniors, tribes, municipalities, and healthcare.

RUS proposed contract modifications to streamline and simplify the timeline for construction and hired 8 additional engineers and loan and grant technicians and allowed some GFRs overseeing larger projects to approve routine contracts to help address the contract approval backlog described above.

Conclusions

BTOP and BIP, as established by the Recovery Act, are intended to promote the availability and use of broadband Internet access throughout the country, as well as create jobs and stimulate economic development. The ability to measure the progress of BTOP and BIP projects and their effects on expanding access to and use of broadband Internet access is an important component of program management and oversight, and could inform future federal programs. NTIA and RUS track the amount of funds disbursed to projects as one measure of progress. However, disbursements do not fully reflect the amount of progress made. Thus, data on non-financial measures of progress, such as the amount of infrastructure deployed, can provide insights into the progress of broadband projects. While NTIA established performance measures and collected data on non-financial measures of progress, RUS did not initially collect comparable data, and once it did begin collecting these data, it could not ensure their quality. Collecting accurate data would enable RUS to better demonstrate BIP projects’ progress toward completion, and outcomes for the BIP program. We also identified challenges that NTIA and RUS both faced in ensuring the accuracy of projects’ subscribership data. Because both agencies have taken steps to improve the quality of the subscribership data reported by awardees, we are not making a recommendation to address this issue in this report.

Recommendation for Executive Action

To ensure RUS is collecting reliable information regarding the effect of investments in broadband, we recommend that the Secretary of Agriculture direct RUS to take steps to improve the quality of its data on the number of fiber miles and wireless access points created by BIP projects.

Agency Comments and Our Evaluation

We provided a draft of this report to the Secretary of Commerce and the Secretary of Agriculture for review and comment. The Department of Commerce provided technical comments that we incorporated into the report as appropriate. The Department of Agriculture provided written comments on a draft of this report, which appear in appendix III. The department disagreed with our characterization that RUS does not collect adequate data to measure the progress of BIP and noted that RUS collects financial data as well as contract-level data with information on planned construction for each project. The department also noted that RUS collects data on the number of subscribers, which it implies is a relevant measure of performance for the BIP program. We agree that RUS collects financial data, but as we note in the report, financial data do not fully reflect the progress of the program. Thus, non-financial data, such as fiber miles and wireless access points deployed, provide an additional indicator of BIP’s progress. As we note in the report, RUS’s data for these measures, as well as the number of subscribers, are unreliable, which hinders RUS’s ability to assess the progress of BIP. Although the department neither agreed or disagreed with our recommendation that RUS should improve the quality of its data on the number of fiber miles and wireless access points created by BIP projects, the department stated that RUS has already taken steps to improve the quality of its data. If RUS takes action and can demonstrate that the data are reliable, we will close the recommendation as implemented.

We are sending copies of this report to the appropriate congressional committees, the Secretary of Agriculture, and the Secretary of Commerce. In addition, the report is available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-2834 or goldsteinm@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix IV.

Appendix I: Scope and Methodology

This appendix provides information on the methodologies that we used to assess (1) the progress made in implementing broadband projects funded by the American Recovery and Reinvestment Act of 2009 (Recovery Act), (2) the effect of the projects on expanding access to and adoption of broadband service, and (3) the challenges that grant and loan recipients face in completing broadband projects, and the actions that agencies are taking to help address these challenges.

Progress of Programs

To determine the progress made in implementing the projects funded by the Recovery Act, we obtained data that allowed us to calculate the amount of funds awarded and the amount of funds disbursed to projects participating in the Broadband Technology Opportunities Program (BTOP) administered by the National Telecommunications and Information Administration (NTIA), and the Broadband Initiatives Program (BIP) administered by the Rural Utilities Service (RUS). In addition, we obtained data from NTIA on the status of BTOP in meeting the targets for the key performance indicators established by NTIA: miles deployed or leased, community anchor institutions connected, workstations installed in public computer centers, and new broadband subscribers. To determine the reliability of these data, we reviewed relevant documentation, including manuals, guidance and forms provided to awardees reporting the data, descriptions of internal controls, and Inspector General reviews of the data from the Department of Agriculture and the Department of the Treasury (which manages the payment system used by BTOP recipients); and we interviewed agency officials about their processes for reviewing the data and ensuring their accuracy. We found the data generally reliable for our purposes of reporting the amount of funds disbursed to BTOP and BIP awardees, and the amount of progress made toward meeting NTIA’s targets for BTOP, although we did note and report on limitations with NTIA’s subscribership data.

Effect of Programs

To assess the effect of the projects on expanding access to and adoption of broadband service, we reviewed data collected by NTIA and RUS, such as the aforementioned data on the progress of BTOP projects in deploying miles, connecting community anchor institutions, installing workstations in public computer centers, and creating new broadband subscribers. Our efforts to determine the reliability of BTOP’s program data are discussed above. We reviewed the data that RUS collects from BIP awardees on the number of subscribers accessing BIP-funded service, but based on conversations with RUS officials, determined the data were not reliable for our purposes. We also interviewed BTOP and BIP awardees to understand the effects of their individual projects. We used the following criteria to select awardees to interview: projects that had received two-thirds of their award (thereby meeting NTIA’s “substantially complete” requirement), large award size, and geographic dispersion.

Challenges

To determine the challenges that grant and loan recipients face in completing projects, we interviewed NTIA and RUS officials, including NTIA’s Federal Program Officers (FPO) and RUS’s General Field Representatives (GFR), who serve as a point-of-contact for the awardees. We also interviewed the BTOP and BIP awardees mentioned in the previous paragraph, as well as some awardees that had not received two-thirds of their award. For BTOP, we reviewed awardees’ progress reports for the first quarter of 2012, which covered the time period from January 1, 2012, to March 31, 2012. In these quarterly reports, awardees must report on their progress toward meeting milestones (progress is based on expenditures), and describe the reason for any variance from their baseline plan or subsequent written updates provided to their program officer. Awardees must also describe any challenges or issues faced in the past quarter in achieving planned progress, and any areas where assistance from NTIA is needed. We created content categories for the challenges reported by the awardees and for whether they requested assistance from NTIA. Two analysts independently coded each response into one of the categories; any discrepancies in the coding of the two analysts were discussed and addressed by the analysts. For BIP, we reviewed a tracking spreadsheet maintained by RUS, in which the GFRs provide the status of each BIP project and in some cases, note issues or challenges facing individual projects.

Agency Actions to Address Challenges

To determine the actions NTIA and RUS are taking to address challenges facing BTOP and BIP awardees, we interviewed NTIA and RUS officials, including FPOs and GFRs. We also reviewed documentation of guidance, training, webinars, and workshops provided by the agencies. We asked the BTOP and BIP awardees we interviewed about their experiences with NTIA and RUS, including whether the awardees had faced challenges and the awardees’ perceptions of NTIA and RUS guidance and resources.

We conducted this performance audit from November 2011 through September 2012 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Analysis of Employment Data Submitted by Program Awardees

This appendix responds to a recurring GAO mandate in the American Recovery and Reinvestment Act (Recovery Act) requiring us to comment and report quarterly on estimates of jobs funded and counted as full-time equivalents (FTE) as reported by recipients of Recovery Act funds. To assess the quality of FTE reporting by awardees participating in the National Telecommunications and Information Administration’s (NTIA) Broadband Technology Opportunities Program (BTOP) and the Rural Utilities Service’s (RUS) Broadband Initiatives Program (BIP), we examined recipient-reported data publicly available at Recovery.gov as of August 1, 2012, for these two programs over the 10 quarters in which they reported FTE data. While Recovery Act recipients’ reporting of FTEs began in September 2009, the first quarter that FTE data were submitted for the BTOP and BIP programs was the first quarter of 2010 for BTOP and the second quarter of 2010 for BIP.

Our assessment of the FTE reporting included interviewing NTIA and RUS program officials familiar with awardees’ recipient reporting and a review of the FTE figures and other recipient reported data. Our matches showed a high degree of agreement between the agencies’ assessments of FTE positions reported and our analyses of information recipients provided on their quarterly reports. Based on our analyses and interviews with agency officials, we determined that the recipient-reported data appeared to be sufficiently reliable for the purpose of providing summary, descriptive information about FTEs and other information submitted on grantees’ recipient reports.

The FTE data reported by BTOP and BIP recipients showed an overall rise in the level of employment activity among the projects. As shown in figure 3, the number of FTEs reported by both programs steadily increased from quarter to quarter in 2010 and 2011 with BTOP continuing to show an increase in the second quarter of 2012 while BIP showed a decrease during the first quarter of 2012, and an increase from that in the second quarter of 2012.

Recipients reported that BTOP-funded FTEs increased from about 20 FTEs in the first quarter of 2010 to a peak quarter of about 4,500 FTEs in the second quarter of 2012. BIP recipients reported an increase from 86 FTEs in the second quarter of 2010 to about 4,500 in the second quarter of 2012, with a peak of over 4,700 FTEs in the fourth quarter of 2011.

Appendix III: Comments from the Department of Agriculture

GAO Comments

1. We do not imply that RUS does not or should have the same data as NTIA. Rather, we note that the agencies have financial and non- financial measures of progress. Both NTIA and RUS have reliable financial measures of progress. However, as we note, the financial data do not fully reflect the amount of work completed. Therefore, non-financial data provide additional insight into the progress of the programs toward completion. In its letter, RUS says that the number of fiber miles is not as relevant as the number of subscribers. But, since RUS began tracking fiber miles and wireless access points deployed, we assume the agency considers these important measures of progress. Yet, at the time of our review, RUS officials told us that they could not ensure the quality of the data. We agree that the number of subscribers is an important performance measure for the programs; however, unlike fiber miles and wireless access points deployed, the number of subscribers is not an indicator of the progress of a project toward completion. Rather, it is an outcome- based performance measure for a completed project, as the number of subscribers depends in part on the deployment of fiber miles and wireless access points. Further, as we discuss in the report, RUS officials told us that its subscriber data are inaccurate. Therefore, at an aggregate, program level, RUS does not have accurate non- financial data to measure the progress of BIP. 2. We acknowledge that RUS tracks the progress of individual BIP projects through its review and approval of contracts. RUS noted that it uses the information provided in contracts to monitor the construction of fiber facilities and the installation of access points on a project-by-project basis. RUS also noted that as a lender, there was no need to track these measures at an aggregate level. However, considering the number of BIP projects, the provision of grants in addition to loans, and the emphasis on transparency and quickly deploying Recovery Act-funded projects, aggregated information provides a holistic view of the progress of BIP. In addition, if the underlying project-level data are reliable, it is unclear why the aggregate level information would be unreliable, as RUS officials told us during our review. 3. We included additional information on RUS’s efforts in the body of our report. 4. We revised our report to note that the quarterly reports also include awardees’ financial statements. 5. We revised our report to identify the additional steps that RUS has taken to help awardees address their challenges, including implementing a detailed review process and providing guidance and assistance to awardees. 6. At the time of our review, RUS officials told us that they could not ensure the quality of the data on the number of fiber miles and wireless access points deployed. RUS officials attributed the problems with data quality to the short collection period, as they had begun collecting these data in June 2012. If RUS takes action and can demonstrate that the data are reliable, we will close the recommendation as implemented.

Appendix IV: GAO Contact and Staff Acknowledgments

GAO Contact

Mark L. Goldstein, (202)512-2834 or Goldsteinm@gao.gov.

Staff Acknowledgments

In addition to the contact named above, Michael Clements (Assistant Director), Tom Beall, Crystal Huggins, Bert Japikse, Aaron Kaminsky, Carol Patey, Amy Rosewarne, Beverly Ross, and Jon Stehle made key contributions to this report.