Trade Adjustment Assistance: Commerce Program Has Helped Manufacturing and Services Firms, but Measures, Data, and Funding Formula Could Improve
Released: 2012-09-13T00:00:00

Letter

Background

The purpose of the TAA for Firms program is to help trade-impacted, economically distressed U.S. manufacturing, production, and service firms make adjustments that may enable them to remain competitive in the global economy. The program delivers technical assistance to firms by developing business recovery plans and providing matching funds to implement the projects in the plans. EDA uses its appropriation for the TAA for Firms program to fund 11 TAA Centers, signing a cooperative agreement with each center. The centers provide assistance to U.S. manufacturing, production, and service firms in all 50 states, the District of Columbia, and the Commonwealth of Puerto Rico. Each TAA Center uses the funding that EDA allocates under the cooperative agreement to cover its administrative and operational costs and works directly with firms in its geographic region to provide assistance on a cost-share basis. None of the program funds go directly to firms; instead, funds go to third- party consultants to implement firms’ projects. The centers cover geographic regions of varying size, composed of one to eight states. A TAA Center may be affiliated with a university, private firm, or nonprofit organization; however, as shown in figure 1, nonprofit organizations currently manage four centers, while the other seven centers are affiliated with universities.

Staff at the TAA Centers generally target their outreach to small and medium-sized firms in the manufacturing, agricultural, and service sectors.firms, including the following: They use a variety of outreach methods to identify potential analyzing industry databases; conducting cold calls and providing information via websites, e-mail, and mail; presenting at seminars and meetings hosted by banking consortia, business and trade associations, and members of Congress; collaborating with state and local entities, including local departments of commerce and agriculture, economic and small business development centers, and chambers of commerce; targeting firms with workers who qualify for the TAA for Workers program or firms involved with International Trade Commission antidumping lawsuits; and acting on referrals from program consultants and current and former clients.

Once a firm connects with a TAA Center, staff may use various methods to determine whether firms are financially able to participate in the program. Some centers review preliminary financial documentation in an effort to help ensure that firms are eligible for assistance and committed to participating in the program. One center developed a series of eligibility questions on its website, so that firms can determine whether they are eligible for the program before applying for assistance.

The TAA for Firms program process has three phases—petition for certification, recovery planning, and business recovery plan implementation—requiring collaboration among firms, TAA Centers, and EDA. Figure 2 presents a flowchart summary of the process.

A firm seeking TAA program benefits must submit a petition to EDA to receive certification of its eligibility to participate in the program. TAA Center staff collect documentation from a firm to demonstrate in the petition each of the following: 1. A significant number or proportion of the firm’s workers have been or are threatened to be totally or partially separated. 2. Total sales and/or production or sales and/or production of a product that represents at least 25 percent of the firm’s total have decreased absolutely during the 12-month period preceding the most recent 12- month period; or total sales and/or production, or that of a product that represents at least 25 percent of the firm’s total, have decreased during the most recent 12-month period compared with the preceding 12-, 24-, or 36-month period. 3. Increases in imports of articles or services comparable to, or directly competitive with, the firm’s articles or services contributed importantly to worker separations (or threat thereof) and the declines in sales or production. Firms must generally demonstrate that import impact has occurred, using a combination of import data or statements from their customers to certify that imports were a factor in the decreased purchase of the firm’s products.

Once EDA approves the petition and certifies the firm, the firm and TAA Center staff have 2 years to develop a business recovery plan and submit it to EDA for approval. Center staff work closely with the firm’s management to identify the firm’s strengths and weaknesses and develop a customized plan designed to stimulate its recovery and growth. EDA requires standard information in each business recovery plan, such as a description of the firm’s competitive problems, prospects for recovery, and specific technical assistance projects. However the length, level of detail, and amount of information provided in the plans vary across the TAA Centers. EDA officials review and approve the plans on the basis of whether they meet regulatory requirements and supply the necessary supporting documents.

A firm that requests $30,000 or less in total assistance to implement an approved business plan must pay at least 25 percent of the cost of the assistance. Firms that request more than $30,000 in total assistance, up to $75,000, must pay 50 percent of the total cost.

TAA Centers proposed four types of projects in firms’ business recovery plans (see fig. 3).

To assist a firm in implementing the projects in its business recovery plan, TAA Center staff work with firm management to identify and competitively select third-party consultants with the specific expertise needed. In some instances, center staff said that they collaborate with consultants from Commerce’s Manufacturing Extension Partnership program to implement projects in the TAA firms’ business recovery plans. (Please see app. II for a discussion of the similarities, differences, and interactions among the TAA for Firms, Manufacturing Extension Partnership, and other Commerce programs.)

In addition to mandating that we report on the TAA for Firms program, the TGAAA mandated that we report on the other TAA programs. In July 2012, we issued our report on the TAA for Farmers program. Our reports on the TAA for Workers and Communities programs are forthcoming.

TGAAA Changes Led to Program Improvements and More Participation

The 2009 legislative changes to the TAA for Firms program resulted in reduced firm certification processing times, new performance reporting, and increased firm participation. However, EDA officials and TAA Center staff said that the lapse of these legislative changes from February 2011 to October 2011 and the uncertainty regarding program funding contributed to a decrease in firm participation in fiscal year 2011.

Some TGAAA Changes Resulted in Reduced Firm Certification Processing Times, New Performance Reporting, and Increased Firm Participation

The TGAAA marked the first major change in the TAA for Firms program since 1986 (see table 1). The 2009 legislation included changes to the program, such as eligibility for service sector firms and more flexible certification requirements. However, TGAAA provided that the provisions that expanded eligibility would expire and that on January 1, 2011 the TAA for Firms program be administered as if the expanded provisions had not been enacted. As authorization of the program was about to expire on January 1, 2011, Congress passed the Omnibus Trade Act of 2010 to extend the program through February 2012. However, this legislation did not reinstitute the TGAAA changes, which resulted in a lapse of the TGAAA changes from February 2011 until October 2011. In October 2011, the TAA Extension Act of 2011 reinstituted many of the TGAAA’s changes, including service sector firm eligibility and the more flexible certification requirements.

Four changes mandated by the 2009 legislation contributed to improvements in program operations and increased participation:

Creation of director’s and other full-time positions. The creation of a director’s position and other full-time positions for the program resulted in reduced certification processing times for petitions. Prior to the 2009 legislation, the TAA for Firms program was administered by rotating staff members and interns. TAA Center staff said that this resulted in inconsistent review of petitions. Center staff reported that with the full-time professional staff now available, reviews are dependable. In addition, EDA stated in its 2011 annual report that its staff reduced the average processing time for petitions to 21 days—a 48 percent reduction from fiscal year 2010 and a faster turnaround than the 40 days required by the TGAAA.

New annual reporting on performance measures. The TGAAA required EDA to gather information on performance measures and submit annual reports to Congress. EDA has submitted three annual reports to Congress on these performance measures as a result of the legislation.

Inclusion of service sector firms. According to our analysis of EDA data, the inclusion of service sector firms allowed EDA to certify 26 firms not previously eligible for assistance in fiscal years 2009 through 2011. Examples of service sector firms assisted by some TAA Centers include architectural engineering firms, telecommunications firms, and software development firms.

Expansion of the “look-back” period from 12 months to 12, 24, or 36 months. Our analysis of EDA data shows that 32 additional firms participated in the program in fiscal years 2009 through 2011 as a result of the expanded the look-back period. Prior to the legislative changes, firms were allowed to compare sales and production data in the most recent 12 months only with data from the immediately preceding 12-month period. Staff at the TAA Centers told us that it was difficult to certify some firms with the 12-month look-back period, because that amount of time was often not adequate to demonstrate declines in sales and production. Center staff said that the expanded look-back period was particularly helpful during the recent recession, because it enabled more firms to demonstrate the requisite declines.

Although these legislative changes generally improved the program and increased participation, TAA Center staff noted some challenges in assisting service sector firms. For example, some service sector firms had difficulty establishing import impact along with requisite declines in sales and production. Because service sector firms do not have industry- specific codes that allow TAA Center staff to analyze import data, EDA allowed the use of certification letters from firm customers to demonstrate import impact in lieu of import data. However, many of the firms’ customers did not want to certify in writing that they were outsourcing services, according to center staff. As a result, staff said that some service sector firms could not provide sufficient evidence of import impact. In addition, some Center staff expressed concern that they did not have sufficient funding to serve both manufacturing and service sector firms.

Participation Increased after TGAAA Changes but Declined after Changes Lapsed and Future of Program Became Uncertain

From fiscal year 2008 through fiscal year 2010, EDA certified and approved an increased number of petitions and business recovery plans (see fig. 4). In addition, with a few exceptions, the TAA Centers generally submitted an increased number of petitions and business recovery plans during this period. According to center staff, the economic downturn contributed to the increase in firms applying for and receiving assistance from the TAA for Firms program from fiscal year 2008 through fiscal year 2010, because more firms could demonstrate a decline in sales and employment. Additionally, EDA officials and TAA Center staff stated that the 2009 legislative changes increased interest in, and demand for, the program by prospective firms. For example, EDA officials and TAA Center staff said the news of the increased funding authorization generated many inquiries from firms, and one TAA Center official noted that the large increase in authorized funding in the TGAAA led to the expectation that the program would be able to service a greater number of firms. EDA officials also stated that demand for the program increased because more firms were eligible under the expanded look-back period.

In total, EDA certified 878 petitions and approved 761 business recovery plans for fiscal years 2008 through 2011. Because TAA Centers use a variety of methods prior to completing a petition to determine if a firm is financially able to participate in the program, center staff told us that EDA approved almost all of the finalized petitions they received. In addition, center staff said that EDA rarely rejected submitted business recovery plans, owing to the firms’ in-depth diagnostics and detailed business recovery plans.

Figure 4 also shows that EDA certified fewer petitions and approved fewer recovery plans in fiscal year 2011 than in fiscal year 2010. Certified petitions decreased from 330 in fiscal year 2010 to 149 in fiscal year 2011, and approved business recovery plans decreased from 264 in fiscal year 2010 to 183 in fiscal year 2011. EDA officials and TAA Center staff attributed the decline to three factors:

Lapse of TGAAA changes. EDA officials and TAA Center staff reported that the lapse of the legislative changes from February to October 2011 meant that service sector firms were no longer eligible to receive assistance. Center staff also said that the lapse in the TGAAA provisions disrupted the certification and approval process for several service sector firms and excluded them from participating. In addition, center staff stated that without the option to use the expanded look-back period, some petitioning firms that would have been eligible before the lapse could not show the requisite declines in sales or production during this time.

Program uncertainty. The President’s fiscal year 2012 budget proposed to eliminate the TAA for Firms program. EDA reported that the uncertainty regarding the program’s future funding caused the TAA Centers to focus on existing clients instead of identifying new firms. One center stated in its 2010 annual report that the possibility that the program might be cancelled required the center to reserve sufficient funds to cover closeout costs rather than bring new firms into the program. Other center staff confirmed that uncertainty about the program affected outreach efforts and budgeting, because of concerns that the program would be eliminated.

Improvement in the economy. Some TAA Center staff said that fewer firms were eligible to participate in the program because the economy’s improvement from fiscal year 2010 to fiscal year 2011 prevented some firms from demonstrating a decrease in employment, sales, and production.

EDA’s Performance Measures and Data Collection Limit Its Ability to Gauge Program Effectiveness, but Our Analysis Shows Positive Impact

EDA collects data on 16 measures reported in its annual report to Congress and in Commerce’s performance and accountability report, but we found that these performance measures, and EDA’s use of them, do not adequately focus on program outcomes. In addition, EDA’s lack of centralized, comprehensive, and reliable data on participating firms limits its ability to improve program management and program outcomes. However, we found the program may have had positive outcomes that are not captured in EDA’s performance measures and data collection. Our data analysis showed that participation in the program was positively associated with an increase in sales, and our survey respondents reported satisfaction with the assistance they received from the program.

Limitations in Performance Measures Hinder EDA’s Ability to Determine Program Effectiveness

As TGAAA requires, EDA currently collects data on 14 performance measures to include in its annual report on the TAA for Firms program, as well as two measures to include in Commerce’s annual performance and accountability report. However, we found that these measures are largely output measures rather than measures of program outcomes. We define performance measurement as the systematic ongoing monitoring and reporting of program accomplishments, particularly progress toward preestablished goals or standards. Performance measures may address inputs: program staffing and resources; outputs: the type or level of program activities conducted or the direct products or services delivered by a program; or outcomes: the results of those products and services.

Outcome-oriented goals and performance measures assess the results of a program, compared with its intended purpose, and are important for ensuring accountability. We characterize most of EDA’s performance measures as output measures, since they measure goods and services delivered by a program—for example, the number of firms certified as eligible for the program or the number of business plans approved by EDA. We characterize only three of the measures— measures 10, 15, and 16—as outcome measures, as shown in table 2.

Outcome measure 10—sales, employment, and productivity of each firm on completion of the program and in each of the 2 years after completion—is closely tied to the program’s intended purpose of helping firms adjust to international trade competition. However, in its 2011 Annual Report on the TAA for Firms program, the primary analysis that EDA offers for its data on performance measures notes how the program’s participants fared relative to the nationwide average for all manufacturing firms. Specifically, employment for participating firms decreased less for TAA for Firms participants than for firms nationwide (a 1.9 percent decrease for program participants, compared with a 4.5 percent decrease nationwide, according to Bureau of Labor Statistics data). Likewise, productivity for TAA for Firms participants increased by less than 1 percent, whereas average productivity increased by 10 percent nationwide since 2009. However, comparing performance data for trade-impacted firms with data for the nationwide manufacturing industry is not sufficient for determining whether the program is effective in helping firms. TAA for Firms recipients are in distress, having already been certified as adversely impacted by international trade. These firms demonstrated lost sales and employees to qualify for the program and cannot be expected to perform at the same level as the national average, which includes firms not impacted by trade competition. Nevertheless, EDA has not attempted to isolate the impact of the program’s assistance on firms from other influences. In November 2009, we reported that using program evaluation methods to rule out plausible alternative explanations for outcomes that may be influenced by a variety of external factors, such as changes in the economy, can help strengthen evaluations. In June 2010, EDA outlined plans for an objective evaluation of the effectiveness and efficiency of the program and individual TAA Centers. However, the evaluation was not implemented because of a lack of funding for it, according to the program’s director.

Several TAA Center officials noted that data from the performance measures do not adequately show the effect that the program has on its participants, and they further commented that they do not find these measures particularly useful for improving program management. For example, a representative from one TAA Center said that an important measure of success is firm survival, which determines the number of jobs that are saved or created, but firm survival is not measured in the annual performance metrics. A representative of another TAA Center noted that a weakness in measuring sales, employment and productivity 2 years after program completion is that the nature of the business cycle is such that sales and production must increase to the point where orders cannot be filled before the firm believes it is justified in hiring new staff. As a result, there is a time lag between increased sales and production and increased employment that might not be reflected in the 2-year data.

EDA acknowledges that the program’s performance measures should be improved and has made improving the measures a goal and taken steps toward it. According to the director of the program, EDA has entered into a partnership with economists from the George Washington University and the University of North Carolina-Chapel Hill to improve the performance metrics for all EDA programs under the Government Performance and Results Act. Planned activities include the creation of a new logic model that will map out inputs, outputs, and outcomes for EDA’s programs. The economists will also assist EDA in creating more expansive performance metrics for all programs, including TAA for Firms. EDA anticipates that this project will be completed by fall of 2014.

Lack of Centralized, Comprehensive, and Reliable Data Affects Program Management and Analysis

Although EDA and the TAA Centers independently collect data on participating firms and program operations that may be useful for evaluating the TAA for Firms program, gaps in centralized, comprehensive, and reliable program data limit EDA’s ability to analyze program trends and inform decisions to improve results.

The issues with EDA’s data that we identified fall into four clear but interconnected categories:

Gaps in centralized data. According to EDA officials, the agency maintains a database of information from petitions, such as firm location, sales or production, employment, and the basis for eligibility. EDA staff also maintain a separate database of information from firms’ approved business recovery plans, including the total dollar amount of assistance and types of projects approved. However, TAA Center staff do not have access to EDA’s databases and maintain their own independent program data and information. We found that EDA does not maintain the necessary data, such as whether the firm is a public or private firm or a multiplant firm, to conduct a quantitative analysis of the effects of TAA for Firms assistance on participating firms, but that the TAA Centers collected the time-series data required to perform an economic analysis. Although we were able to conduct an analysis of the program’s impact on firms, doing so required us to compile more complete and comprehensive firm-level data that the centers had collected for other purposes. For example, in the absence of centralized program data, we utilized the firm-level data cited in the conference report on Commerce appropriations compiled by the centers for fiscal year 2012, including firms’ current and prior sales and employment and the fiscal years in which firms completed projects in their approved business recovery plans.commented that the program would benefit from having a complete data set on firms to respond to future data requests from stakeholders and analyze the effectiveness of the program; however, the data EDA currently has accessible are not sufficient for such an analysis.

EDA makes multiple data requests of TAA Centers. EDA frequently makes multiple requests to the TAA Centers for program data when preparing required reports, and center staff noted that they are often asked to provide or validate program data and information they have already reported, including the data for the program’s annual reports. Staff at several centers that we visited stated that they were able to comply with repeated requests for data, but they cited concerns about the use of limited staff time to prepare reports of data they had previously reported.

Data requests require verification but can still result in inaccuracies. In addition, EDA relies on each of the TAA Centers to validate its data. However, when we compared EDA’s data with data provided by the centers, we identified errors in EDA’s data. For example, we found that EDA’s certification data did not include 30 of the 32 firms that were certified on the basis of the expanded look-back period in fiscal years 2009 to 2011. EDA officials also could not ensure the reliability of data on petition and business recovery plan approvals prior to fiscal year 2008, because EDA had not validated these data with each of the centers.

Lack of guidance results in dissimilar information across TAA Centers.

EDA has not developed guidance on the format and types of program data that TAA Centers should collect, which has contributed to a lack of comparable data on program activities across the centers. Though TAA Centers have begun to track expenses by specific program activities—such as outreach, petition development, and business recovery plan development and implementation—EDA has not clarified how the centers are to record these activities, according to EDA officials. As a result, Commerce’s Office of Inspector General reported that TAA Centers do not appear to be consistent in how each records and allocates costs among program activities. Many of the centers conduct customer satisfaction surveys, but each has its own set of questions and method of administering the survey. Without consistent information, EDA cannot monitor activities across TAA Centers or conduct a comparative program analysis.

EDA officials noted that they had been working toward establishing a centralized database, accessible online, in which TAA Centers and EDA could enter and verify program information. According to officials, this effort is currently on hold pending Commerce’s response to guidance from the Office of Management and Budget (OMB) on the establishment of shared information technology services for federal agencies. Further, EDA headquarters experienced a computer virus during our review, which demonstrated the value of the centers’ systems for obtaining and reporting historical data. However, without centralized comprehensive and accurate data on program operations, EDA is hindered in its ability to effectively evaluate the program, such as by determining the effects of program assistance on firms.

Our Analysis Shows Participation in the TAA for Firms Program Is Statistically Associated with Increases in Firm Sales and Productivity

Given the weaknesses we found in EDA’s performance measures and data collection, and because few other studies have examined the effectiveness of the program, we undertook further analysis to determine the impact of the TAA for Firms program. We found that participation in the program was associated with increased sales, although other factors, such as a firm having multiple plants, had a stronger effect on performance. We used a regression analysis to determine the sales performance of firms that participated in the TAA for Firms program from 1998 through 2011, both before and after program participation, while controlling for other variables, such as the size of the firms and conditions in the economy. By comparing firm sales before and after participation in the program, rather than comparing firms in the program with firms that did not receive assistance, we were able to examine whether participation in the TAA for Firms program was associated with positive outcomes for firms that actually received assistance under the program.

We collected data from the TAA Centers on firms’ sales and employment 2 years prior to having a business recovery plan approved and 2 to 3 years after, along with industry information. Our data included about 570 firms in more than 250 5- and 6-digit North American Industry Classification System (NAICS) industries that had experienced a decrease in either sales or employment owing to import penetration.

About 85 percent of the firms in our analysis were small to medium sized. About one-third had fewer than 25 employees, and about two-thirds had fewer than 50 employees; only about 5 percent had more than 300 employees. We also created overall industry and macroeconomic variables using Bureau of Labor Statistics and Census data, to isolate the effect of the program on firm sales while accounting for other factors in the general economy. We found that some firm- and market-related factors had a stronger and more significant effect on firm sales than did participation in the TAA for Firms program, such as whether the firm was a multiplant firm, and whether it was a publicly or privately held firm. With regard to the effect of program participation on sales, we determined the following:

There is a small positive and statistically significant relationship between program participation and sales. Overall, we estimate that the effect of participation in the program was an increase in firm sales, ranging from 5 to 6 percent on average, if all other factors are held constant. This translated into an average increase of about $280,000 to $350,000. The effect was greater for the firms with 300 or fewer employees that accounted for 95 percent of the firms in our sample. Using productivity (firm sales divided by employment) as one outcome variable, we also found that the effect of the program on productivity was about a 4 percent increase.

As imports rose, sales declined for TAA for Firms clients. Our analysis shows that import penetration was highly statistically significant and most likely had a very negative effect on firm sales. According to our estimates, for every 1 percentage point increase in the industry import penetration ratio, sales of firms included in our analysis decreased by about 16 percent on average. Our analysis also shows that import penetration increased from an average of 34 percent in 2000 to 39 percent in 2011 for industries associated with the firms in our study.

TAA for Firms participation combined with market growth increased firm performance. We found a statistically significant and positive effect of industry market growth on firm sales after firms participated in the program.the percentage change in firm sales increased as market growth increased. For firms in relatively high-growth industries, such as certain types of metal manufacturing, plastic pipe manufacturing, and flooring industries, the combination of participation in the program and industry growth affected sales more positively, with such firms experiencing a 6 to 10 percent increase in sales. This result suggests that firms that participated in the TAA for Firms program during the 1998 through 2011 period were better able to take advantage of growth in their markets or to translate overall market growth into firm sales, compared to before participating in the program. For low- growth markets, the effect was not as positive. Specifically, firms selling products in the lowest-growth (or negative-growth) markets— the bottom percentile of our sample’s industry growth ranges—still experienced declines in sales ranging from -0.5 percent to -2.8 percent. Since our analysis captured marginal effects, this may simply mean that the negative factors weighing on the firm outweigh the positives, including the effects of program participation.

TAA for Firms Clients Reported Satisfaction with Program and Its Outcomes

Our survey of TAA for Firms participants also showed that the program had a positive effect. We conducted a survey of 163 firms that had a recovery plan approved in fiscal year 2009 to obtain their views about their experience with the program; we received responses from 117 of the 163 firms, with a final response rate of 72 percent. The survey included questions about the TAA Center, the consultants who carried out the projects included in the business recovery plans, and the outcomes of the firm’s participation in the program.

More than 90 percent of responding firms reported that they were either very or generally satisfied with the services they received from the TAA Center and the consultants who performed work for them (see fig. 5). Over 80 percent reported that the program helped them to identify projects and business process improvements, and 62 percent said that the program helped them to identify management weaknesses.

In terms of outcomes, survey results indicate that responding firms believed the program was particularly helpful in improving marketing and sales (84 percent); helping them to stay in business (82 percent); helping to improve profitability (73 percent); helping them to retain employees (71 percent); and helping them to hire new employees (57 percent). In narrative responses to our survey’s open-ended questions, 22 firm representatives said that the program helped their business to grow or improve. In addition, 30 respondents wrote positive comments about the TAA Centers’ attentiveness to their needs and the ease of working with the centers.

Only 34 percent of respondents wrote that the program helped their firm with export sales, although we heard in interviews with staff at several TAA Centers that they encourage small firms to increase exports. One survey respondent noted, “TAA Centers should undertake a program which would encourage small businesses to export. In my experience most small businesses do not export because they believe (incorrectly) that exporting is difficult.” Our survey respondents also suggested that the program could be improved by increasing the availability of funding, and allowing funds to be used for capital improvements. (For the complete results of our survey, please see GAO’s e-supplement, GAO-12-935SP.)

During the course of our work, representatives of firms and the TAA Centers identified the following specific ways that the program had helped firms.

In California, a metal parts manufacturer was certified in 2007 because of increased competition from Taiwan and China. The firm faced a 15 percent decline in employment over a 2-year period before certification. The TAA Center helped improve the firm’s manufacturing technology, enabling it to produce more high-end products, increase production efficiency, and lower costs. The Center also helped the firm to develop a marketing strategy and website upgrade to improve the firm’s name recognition. This assistance helped the firm increase its sales by about 8 percent, and productivity has more than doubled. Over the past 2 years, the firm reported that it has hired two additional employees.

In North Carolina, a TAA Center assisted a manufacturer of sheet metal roofs and artisan works with financial consulting and a website upgrade. The company reportedly hired an additional three employees since it began working with the TAA Center and was able to employ more crews than it could previously. The owner said that his company might have had to lay off half its staff without the TAA Center’s help.

In New York, a TAA Center helped a teak furniture manufacturer that sold its products through catalogues. The firm was impacted by competition from imports. The TAA Center encouraged the firm to rebrand itself in order to penetrate higher-end markets through a higher-quality catalogue and targeted advertising. As a result, the firm reportedly doubled sales and hired 16 new staff.

In Massachusetts, a TAA Center helped an environmental management firm that faced growing competition from international trade. The TAA Center provided $10,000 for an improved website, which, according to the president of the company, resulted in a 10 to 15 percent increase in sales in the first year.

EDA’s Allocation of Program Funds Falls Short of Meeting Key Standard for Funding Formulas

EDA allocated funding to the 11 TAA Centers for cooperative agreement years 2008 to 2011 using a funding allocation formula that comprises a set of weighted factors; however, the formula does not take into account the potential number of firms in need of the program and differences in costs across the centers. According to beneficiary equity—a key standard for designing and evaluating funding formulas—funds should be distributed to regions according to the needs of their respective populations and should take into account the costs of providing program services, so that each service area can provide the same level of services to firms in need. However, TAA Centers varied considerably in their costs and use of the allocation they received. Though EDA deobligates and reallocates any unspent funds, it uses its allocation funding formula to do so, thus perpetuating the deficiency of failing to consider variable needs and costs in allocating its funds.

EDA Uses Funding Formula to Allocate Program Funds to TAA Centers

The TAA for Firms authorizing legislation does not specify how EDA should allocate funding to the TAA Centers. In September 2003, EDA developed, in consultation with the centers, a funding allocation formula that it uses to allocate grant funds to each center. EDA intended that the funding allocation formula would develop consistency in the yearly allocation process and provide TAA Centers with sufficient funds to operate a productive program. EDA recognized that such consistency in funding allocations was necessary, in part because of the complexity of the program and the provision of the centers’ professional business advice to assist in the firms’ recovery planning and projects. EDA’s funding formula divides two-thirds of allocated funding equally among the 11 centers according to base funding and two fixed factors:

Geographic size: The TAA Center’s service region in square miles

Number of firms: The service region’s share of the nation’s firms in the agricultural, mining, and manufacturing sectors The funding formula divides the remaining one-third of allocated funding among the TAA Centers according to three variable factors:

Approved business recovery plans: The center’s share of the total number of business recovery plans approved by EDA within the past 2 fiscal years

Employees in approved recovery plans: The center’s share of the total number of employees in the business recovery plans approved in the last 2 fiscal years

Firms achieving expected results: The center’s share of the total number of firms that reported achieving anticipated outcomes from actions the firms took as a result of the program assistance they received during the past fiscal yearOnce it had determined the funding formula factors and measures for each, EDA weighted the factors to determine how it would distribute annual funding to the 11 TAA Centers. The pie chart in figure 6 shows how the funding formula weighs each of the different factors. The bar chart in figure 6 shows the resulting allocation to each center based on each of the factors for cooperative agreement year 2011. For a discussion of the data used to produce the allocation and of the resulting allocation, see appendix IV.

The TAA for Firms funding formula places an emphasis on an equal distribution of program funding across the centers, as it allocates about two-thirds of program funding using base funding and relatively fixed factors (see fig. 6). As a result, TAA Centers’ share of total program funding has remained relatively equal and constant over cooperative agreement years. For example, during the cooperative agreement years 2008 to 2010, each center’s portion of total funding ranged from 7.2 percent to 10.2 percent. Between the 3-year cooperative agreement period of 2008 to 2010 and the 1-year cooperative agreement period of 2011, the change in TAA Centers’ share of total program funding ranged from 0.2 percent to 1.6 percent.

EDA included the formula’s three remaining factors to account for differences in TAA Centers’ performance.

EDA included each center’s share of the total number of firms that reported achieving anticipated outcomes from actions they took as a result of the assistance they received during the past fiscal year and the total number of business recovery plans approved within the past 2 fiscal years.

To offset the potential of those factors to motivate centers to increase their individual funding levels by targeting a large number of very small firms, EDA also included as a final factor each center’s share of the total number of employees in the business recovery plans approved in the past 2 fiscal years.

Funding Allocation Formula Does Not Take into Account Differences in Program Need

EDA’s TAA for Firms funding formula does not include a direct measure of the number of firms potentially in need of the program based on the program’s key objective of providing technical assistance to firms that have lost sales and employment because of increased competition from imports. As a result, the formula falls short of a key criterion that we previously established for the evaluation of allocation formula approaches. Under the beneficiary equity standard, a funding allocation formula should include a factor that distributes funding to each service area according to the respective target population needing assistance so that each service area can provide the same level of services to the population in need. To meet this equity standard, the formula should use reliable and appropriate measures of need in each state or region. Consequently, TAA Centers that may have a greater number of distressed firms because of import competition potentially receive similar amounts of funding as centers serving a much smaller number of trade- impacted firms. For example, there are wide differences in the numbers of certified petitions and approved business recovery plans among the centers. In addition, staff at some centers stated that they undertake outreach efforts to identify firms. Staff at other centers stated that they identify client firms without conducting much outreach, and still others have so many current clients that they do not seek new ones. However, the program lacks information to determine whether it is achieving equity for trade-impacted firms across TAA Centers’ service regions.

Funding Allocation Formula Does Not Take into Account TAA Centers’ Varying Costs

EDA’s allocation of funding also does not take into account variations in TAA Centers’ costs of providing firms assistance. To meet the beneficiary equity standard, a formula should account for differences in the cost of providing services in each region, so that each firm may receive the same level of assistance. The centers provide EDA with information on programmatic costs by cost categories. However, we found that TAA Centers’ direct and indirect operating costs varied considerably during the cooperative agreement years 2008 to 2010 (see fig. 7). For example, during that time period, centers’ costs for personnel salaries and fringe benefits ranged from approximately $877,000 to $2.1 million, and centers’ costs for travel and for equipment and supplies ranged from $52,000 to $260,000.

In addition, indirect cost rates are set according to OMB guidelines and vary depending on whether the TAA Center is affiliated with a university or an independent nonprofit organization.

For university-affiliated centers, indirect costs tend to be higher than those of nonprofit-affiliated centers. The indirect costs for the university-affiliated centers ranged from 12.9 percent of total costs to 30.3 percent of total awarded funds, or about $473,000 to $958,000, for cooperative agreement years 2008 through 2010.

For nonprofit-affiliated centers, the indirect costs ranged from 5.5 percent to 14.1 percent of total awarded funds, or about $251,000 to $662,000, for cooperative agreement years 2008 through 2010 (see fig. 8).

The direct and indirect costs of operating the centers to provide assistance to firms affect the amount of program funding that centers have available for direct assistance to firms. During cooperative agreement years 2008 through 2010, the share of program funding for third-party consultants implementing projects in firms’ approved business recovery plans varied widely across the centers, ranging from 21.2 percent of total expenditures or $670,000 for the Western Center to 73.7 percent or $3.4 million for the New England Center.

In part because of the variation in the direct and indirect costs of operating the centers to provide assistance to firms, marked differences exist in the centers’ backlogs of approved but unfunded assistance to firms. The TAA for Firms program’s total backlog of unfunded assistance approved from cooperative agreement year 2008 to April 2012 exceeds $24 million for a total of 796 firms, but this backlog ranged from $1 million for 27 firms assisted by the Western Center to $3.9 million for 133 firms assisted by the New England Center (see table 3).

EDA’s approach to allocating funding resulted in differences in the centers’ use of the allocations they received. Five centers spent their entire funding allocation by the conclusion of the 3-year cooperative agreement period 2008 through 2010, while six centers did not. The total proportion of unspent funds among the six centers ranged from 1.3 percent to 9.8 percent. In addition, some centers that spent their full allocation had a backlog of unfunded assistance for projects approved since fiscal year 2008, totaling more than $3.4 million as of April 2012. At the same time, some centers that had unspent funds had smaller backlogs totaling less than $1.2 million. Although EDA deobligates any unspent funds and reallocates these funds for the following cooperative agreement period, it uses its allocation funding formula to do so, thus perpetuating the deficiency of not including centers’ variable needs and costs in allocating funds among them.

To meet the beneficiary equity standard, a funding allocation formula should use reliable and appropriate measures of the cost of providing services in each region. Because EDA’s funding formula does not take into account variations in TAA Centers’ costs of providing assistance to firms, EDA cannot ensure that trade-impacted firms in different service areas receive the same opportunities for assistance through the centers. The available evidence we analyzed suggests wide variation in the number of firms that the centers are able to assist and the amount of funding that they may provide to implement approved business recovery plans, raising questions about whether limited program funding is being used effectively.

EDA Is Undertaking Efforts to Revise Its Funding Allocation Formula

EDA officials have begun discussions with TAA Center staff to revise the program’s funding allocation formula. In February 2012, EDA convened a conference of center directors that included a discussion of potential changes to the formula. According to EDA officials, these discussions are still in the early stages and there have been no specific proposals of the factors and weights that might be included in a revised formula. The director of the TAA for Firms program stated that EDA’s current effort to improve performance measures for all of its programs may identify measures that could be used in a revised funding formula. Although EDA officials stated that they recognize the importance of including a factor in the formula that would account for potential program need, they have not identified data that would enable them to measure and account for differences in the numbers of trade-impacted firms across the centers’ service regions. The agency is considering using Commerce’s Census Bureau databases to explore the possible use of available data in a revised funding allocation formula. It is unlikely that any new formula would be finalized before July 2014, according to EDA officials.

Directors at two TAA Centers we visited stated that they have undertaken efforts to reduce their centers’ operational costs in an effort to provide additional funding to implement firms’ projects. One center director noted that the center’s management initiated cost-reduction efforts, even though EDA’s funding formula does not provide incentives to reduce such costs. Several center directors further noted that, since the centers’ operational costs are relatively fixed, any reduction in costs results in an increased amount and proportion of funding to assist firms. The data we analyzed support this inference. For example, although funding for the program remained at $15.8 million in fiscal years 2009 and 2010, the centers helped 114 more firms petition for certification and gained EDA approval for 93 more business recovery plans in fiscal year 2010 than in fiscal year 2009. In addition, for cooperative agreement years 2009 and 2010, the centers increased the total portion of awarded funding for consultants by about 12 percent while reducing costs in other categories.

In revising EDA’s funding allocation formula, the agency and the TAA Centers will likely need to consider how to strike a balance among several key factors—need, costs of providing services, and available resources. Revising EDA’s funding formula presents challenges and risks, which will require flexibility in implementing any allocation under a new formula. For example, center directors stated that a revised formula should be carefully designed to avoid unintended incentives and should encourage centers to reduce indirect costs and maximize the funding available to assist firms. In addition, because revising the formula will likely result in decreased funding for some centers and increased funding for others, any change to the funding formula should include a transition period so that funding recipients have time to adjust, as our prior work has shown.

An abrupt reduction in funding levels could disrupt a center’s ability to provide assistance to firms. Flexibility in transitioning to a new funding formula would allow centers greater predictability and stability to develop long-range plans and meet their current commitments.

Conclusions

The United States has arguably gained much from its engagement in the global economy and its active pursuit of trade liberalization. Lowering trade barriers such as tariffs offers benefits to consumers and creates new opportunities for American exporters abroad. Yet these long-term, widely disbursed gains are also accompanied by adjustment costs borne directly and in a more concentrated manner by import-competing firms and their workers. Over the past decade, the pace of trade liberalization has been vigorous, as has been the growth in U.S. imports. Although funding for Commerce’s TAA for Firms program, at less than $16 million, is small relative to the $1.3 trillion rise in imports over the past decade, our economic analysis and survey results show that the program has delivered positive results for participating manufacturing and services firms. We found that these firms receive individual attention from TAA Center professionals located in their regions, practical help in developing business recovery plans, and federal matching funds to pursue projects designed to address competitive weaknesses and capitalize on strengths. Many participating firms that were negatively impacted by trade have recorded gains in sales and productivity since starting the program. Our analysis of more than 500 firms’ financial data and other firm and economic factors shows that this sales growth is positively associated with participation in the TAA for Firms program.

The changes to the TAA for Firms program that Congress enacted in the TGAAA in 2009 gave EDA and TAA Center officials more flexibility in certifying firms, strengthened professional management of the program, and improved transparency regarding the program’s performance. However, enhanced accountability can be accomplished only through better measures of how the program is helping firms adjust to import competition. EDA collects performance data, but few of its performance measures are outcome oriented. EDA also has several interrelated weaknesses in its collection of data that make needed reporting burdensome and unreliable at times. Better and more readily retrievable data would give EDA and Congress a more comprehensive and complete picture of program activities and enable more meaningful and ongoing analysis of impact. Given its small budget relative to the demonstrated need for trade adjustment assistance—as suggested by the substantial backlog of approved but unfunded projects—EDA can do more to ensure that its allocations reflect firms’ and regions’ varied needs for assistance and TAA Centers’ varied costs in providing this assistance. EDA can also encourage more efficient program administration by making the cost of services a criterion in its funding formula and by incentivizing TAA Centers’ cost-containment efforts, so that more funds are available to serve firms. EDA has recognized many of these weaknesses and has made initial efforts to address them. Given the current pursuit of further trade liberalization, following through with these improvements to the TAA for Firms program is essential to ensure that the program uses its budget in the most efficient manner possible.

Recommendations for Executive Action

We recommend that the Secretary of Commerce take the following three actions: 1. To ensure that the performance measures used to evaluate the TAA for Firms program demonstrate program results and to help ensure that EDA can comprehensively evaluate the effectiveness of the program, broaden the program’s evaluation approach, for instance, by developing additional quantifiable outcome-oriented performance goals and measures for key program areas and conducting further analysis of the data to isolate the impact of the TAA for Firms program from other influences, such as economic trends. 2. To improve the data available to manage and evaluate the TAA for develop a data system to consistently collect, maintain, and analyze sufficiently reliable and up-to-date data on program operations and participant firms. 3. To ensure that EDA’s allocation of funding to TAA Centers reflects varied program needs and costs, revise the program’s funding formula by reevaluating the factors and weights it uses to allocate funding under its cooperative agreements, and include measures of need, such as the number of import-impacted firms in each center’s service region and the center’s costs in providing assistance to firms.

Agency Comments

We provided a draft of this report to the Department of Commerce for comment. We received written comments from Commerce, which are reprinted in appendix V. Commerce concurred with our findings and recommendations and provided additional information and observations on implementing our recommendations. For example, Commerce stated that EDA intends to focus on developing improved performance measurement and accountability. In addition, Commerce noted that EDA intends to take steps to improve its data collection and examine the funding allocation formula used to distribute program funds to the TAA Centers in collaboration with the centers and Congressional stakeholders. Commerce expects to complete these efforts by 2014.

We also received technical comments from Commerce, which we incorporated as appropriate.

We are sending copies of this report to the appropriate congressional committees, the Department of Commerce, and other interested parties. In addition, the report is available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-4101 or gomezj@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix VI.

Appendix I: Scope and Methodology

The Trade and Globalization Adjustment Assistance Act of 2009 (TGAAA), part of the American Recovery and Reinvestment Act of 2009, mandated that we report on the operation and effectiveness of the Trade Adjustment Assistance (TAA) for Firms program. We examined (1) the results of the legislative changes on program operations and participation, (2) the performance measures and data that EDA uses to evaluate the program and what these tell us about the program’s effectiveness, and (3) how program funding is allocated and spent.

To determine the results of the legislative changes on the program’s operations, we reviewed relevant program legislation and regulations as well as agency documentation and data on the number and type of firms participating in the TAA for Firms program. We reviewed program regulations and agency guidance outlining the operational and administrative changes to implement the amendments under the TGAAA. We collected from the Department of Commerce’s (Commerce) Economic Development Administration (EDA) and the 11 TAA Centers’ data on certifications and approved adjustment plans from fiscal years 2008 to 2011. We analyzed these data and the data reported in the TAA for Firms annual reports for fiscal years 2009 through 2011 to determine the types of services firms received and how the legislative changes impacted program participation. We assessed the reliability of the data by checking for accuracy and completeness. We determined that the data were sufficiently reliable for the purposes of reporting the number of submitted and approved petitions and approved business recovery plans. We interviewed EDA officials, including the TAA for Firms program Director, Chief Counsel, and certifying officials, to determine the effects of the legislative changes on the program. We also conducted a site visit or teleconference with each of the 11 TAA Centers, interviewing staff, representatives of participant firms, and consultants to understand procedures for identifying and assisting eligible trade-impacted firms and to obtain their opinions and observations on the legislative changes’ effects on the program.

We conducted site visits to eight TAA Centers—in Ann Arbor, Michigan; Atlanta, Georgia; Blue Bell, Pennsylvania; Boulder, Colorado; Chicago, Illinois; Los Angeles, California; North Billerica, Massachusetts; and Seattle, Washington. We conducted teleconferences with the remaining three centers—in Binghamton, New York; Blue Springs, Missouri; and San Antonio, Texas. We selected our site visit locations to achieve geographic diversity and a mix of nonprofit and university-affiliated centers, as well as a range of industries, numbers of submitted and approved petitions, and numbers of business recovery plans. To determine how the TAA for Firms program relates to other economic development programs that assist manufacturers, including Commerce’s Manufacturing and Extension Partnership (MEP) program, we interviewed TAA Center staff, MEP consultants, and officials at Commerce headquarters. We also reviewed documentation of program objectives, eligibility requirements, and services provided.

We assessed EDA’s performance measures and data for the program. We analyzed the two performance measures reported for the TAA for Firms program in the agency’s performance and accountability reports as well as the 14 measures specified by TGAAA and included in the TAA for Firms annual reports. Using criteria from prior GAO work, we assessed the extent to which the program’s performance measures reflect the characteristics of effective performance measures. We reviewed other reports about the program: one conducted by the Urban Institute in November 1998 and our prior report on the program. To examine how program data are collected and used to manage the program, we interviewed EDA officials and TAA Center staff and reviewed EDA and TAA Center program data and reports. We interviewed EDA officials responsible for collecting and reporting program data, as well as TAA Center staff, to determine the procedures for collecting, validating, and reporting data on program operations. We also reviewed data and documents, including quarterly and annual reports from EDA and the centers, to understand the types of program data collected and the purposes for which the data are used.

To estimate the impact that the TAA for Firms program assistance has had on firm performance, we collected data on a group of participant firms with business recovery plans approved in fiscal years 2009 through 2011 and data on general economic indicators, and we conducted a regression analysis with these data to assess the effects of the assistance on this group of participating firms, controlling for firm, industry, and economywide factors. From each of the 11 TAA Centers, we obtained financial data on firms that had an approved business recovery plan and had completed at least one project in their approved recovery plan in fiscal years 2009, 2010, and 2011. We assessed the reliability of the data we received from the centers by testing the data for obvious errors and completeness. In addition, we interviewed and received written responses from staff at the centers who had knowledge of the data, to obtain information on the procedures for collecting and verifying the data reported. The centers drew their responses to our data collection instrument primarily from the petition and business recovery plan approval data they collect from firms. In doing so, they used the data that they had previously compiled for a conference report on Commerce appropriations for fiscal year 2012. This helped ensure the accuracy, completeness, and consistency of the data. We found the data sufficiently reliable for the purposes of our analysis. For our dependent variable or outcome variable, we used data on yearly firm sales during this period as a proxy for firm performance. We combined these data with data on industry-wide and firm-specific variables that determine performance, such as firm-specific financial data, growth rates, firm size, age, and other qualitative firm and industry indicators. To control for factors in the general economy, we incorporated macroeconomic variables in our analysis, including the Census Bureau’s regional unemployment rates and a yearly time trend. In addition, to estimate the effects of trade on firm performance, we calculated and incorporated an international trade variable—the import penetration ratio for each firm’s industry. Our regression model determined whether participation in the TAA for Firms program had a statistically significant effect on firm performance after the firm received assistance under the program. Because the data we collected did not include any information about firms that did not participate in the program, this analysis allows us to make inferences from the regression estimates only for the firms that participated in the program during this time period. We present more complete information about the data and methodology of the analysis in appendix III.

To report the views of certified firms on program operations and on the quality and impact of program services, we interviewed firm representatives and consultants during our site visits to TAA Centers. We also conducted a survey of firms that had a business recovery plan approved by the Trade Adjustment Assistance for Firms program in fiscal year 2009. We selected this population to ensure that those surveyed had received assistance from one of the TAA Centers and had some experience with implementing a recovery plan. Our research at the beginning of our review indicated that it would be feasible to survey these firms within our time frame and achieve an acceptable response rate. We conducted formal pretests with representatives of three firms and considered comments from EDA officials to ensure that our questions were appropriate and could be understood by respondents. Our survey covered the following topics:how the firms learned of the program, the types of assistance the firms received from the TAA Centers, the level of the firms’ satisfaction with the assistance, and the impact of the assistance on the firms. We administered our survey between March 2012 and April 2012. We surveyed the 163 firms for which we obtained contact information, of the 171 firms that had a business recovery plan approved in fiscal year 2009. We received responses from 117 of the 163 firms, with a final response rate of 72 percent. However, because we did not randomly select the firms we surveyed, the survey results do not permit us to draw conclusions about all firms participating in the TAA for Firms program. We conducted an analysis of our survey results to identify potential sources of nonresponse bias by comparing respondents to nonrespondents on three key characteristics: total sales, number of employees, and total amount of technical assistance approved. This analysis did not indicate any large differences between respondents and nonrespondents for these three characteristics, and we determined that the survey results for the 117 respondents were sufficiently reliable to present in our report. The survey and a more complete tabulation of the results can be viewed online at GAO-12-935SP.

To examine the funding formula EDA uses to allocate program funds and to examine how funds were spent, we reviewed EDA’s 2003 spending plan, which defines the factors included in the formula as well as the data used to measure each of the factors. In addition, we analyzed EDA’s available spending plans covering cooperative agreement years 2009 through 2011 to assess the measures used to allocate program funding to the TAA Centers during each of those cooperative agreement years. We used equity standards from social science research for evaluating and designing funding allocation formulas to assess the TAA for Firms formula. To determine the amount of funding allocated to each of the 11 centers, as well as the centers’ direct and indirect costs, we analyzed EDA’s budget data for cooperative agreement years 2008 through 2011. We also analyzed centers’ data on the amount of approved, unfunded assistance for cooperative agreement years from fiscal year 2008 to April 2012, and the number of firms affected. We reviewed published work of the Commerce Office of Inspector General’s assessment of the administrative costs of the centers.

We conducted this performance audit from July 2011 to September 2012 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Trade Adjustment Assistance for Firms and Other Department of Commerce Programs

TAA for Firms and Manufacturing Extension Partnership Program

The MEP program, like the TAA for Firms program, is a Commerce- administered program aimed at helping manufacturers. The MEP program was established in 1988 and is administered by Commerce’s National Institute of Standards and Technology (NIST) to enhance productivity and technological performance and to strengthen the global competitiveness of small and medium-sized U.S. manufacturers, helping them create and retain jobs. Under this program, NIST has established relationships with 60 nonfederal organizations, called MEP centers, located throughout the United States and Puerto Rico. NIST enters into annual cooperative agreements with each of the 60 MEP centers, whereby NIST provides federal funding to the centers subject to the MEP centers’ providing matching funds from state and local entities and meeting performance measures. These centers provide services to small and medium-sized manufacturers to help them develop new customers, expand into new markets, and create new products. MEP centers focus on helping manufacturers in five key areas––technology acceleration, supplier development, sustainability, workforce, and continuous improvement. Specifically, MEP centers enter into contracts with firms to deliver technical assistance to improve their manufacturing processes and productivity, expand capacity, adopt new technologies, utilize best management practices, and accelerate company growth. Table 1 provides a comparative overview of the TAA for Firms and MEP programs.

As shown in table 4, trade-impacted firms can qualify for either program, although the eligibility requirements for each program differ. For example, the TAA for Firms program requires applicants to complete a certification process that demonstrates trade impact on the firm’s employment, sales, and production. In contrast, the MEP program does not require applicant firms to go through a certification process. Firms must only demonstrate they are a manufacturer under industry-specific codes, according to MEP officials. Officials from the TAA for Firms program said that in some instances, trade-impacted firms may choose to participate in the MEP program rather than the TAA for Firms program because they can receive faster assistance. In addition, officials confirmed that participant firms in the TAA for Firms program have subsequently participated in the MEP program for additional services, and vice versa. However, officials from both programs did not have data to determine the extent to which this has occurred.

Although trade-impacted firms can qualify for both programs, and some trade-impacted firms have received assistance from both programs, TAA for Firms and MEP program officials said that the programs are more complementary than duplicative in nature. For example, the programs collaborate at the state, regional, and national levels to provide services to manufacturing firms. At the state and regional levels, MEP consultants often bid on and win contracts to implement projects in the TAA participant firms’ business recovery plans, according to TAA Center and MEP staff. Staff from both programs also said that MEP consultants sometimes refer firms to the TAA for Firms program and that TAA for Firms officials sometimes refer firms to the MEP program. At the national level, EDA officials informally consult with MEP officials at NIST on best practices and opportunities for collaboration. For example, EDA staff said that they have spoken at MEP conferences to educate consultants about the TAA for Firms program.

TAA for Firms and Commerce’s Economic Adjustment Assistance Program

In addition to administering the TAA for Firms program, EDA administers the Economic Adjustment Assistance (EAA) program through six regional offices. The EAA program’s purpose is to help economically distressed communities by supporting a wide range of construction and other assistance using flexible tools to address pressing economic recovery issues. TAA Center officials said they had little interaction with EAA officials or the EDA regional offices because the EAA program assists communities rather than firms.

Appendix III: Economic Analysis of the Effect of Trade Adjustment Assistance Policy on Firm Market Performance

We examined the impact of the TAA for Firms program on participant firms’ performance and effectiveness using statistical analysis. For this analysis, we obtained a sample of financial data on firms from the 11 TAA Centers that had an approved business recovery plan and were certified from 1998 through 2010. We used regression analysis to assess whether there was a statistically significant relationship between firm performance and participation in the TAA for Firms program, controlling for other firm- specific, industry-specific, macroeconomic, and trade factors. Using firm sales and firm productivity as proxies for firm performance, we found a positive and statistically significant relationship between participation in the TAA for Firms program and the performance of the firms in our sample. Using different model specifications, we estimated that the firms participating in the TAA for Firms program experienced yearly increases in sales of 5 to 6 percent, all else held constant. Our results also suggest that participation in the TAA for Firms program was associated with increased firm productivity, measured by the ratio of yearly sales to employment, of about 4 percent per year, all else held constant. Moreover, we observed higher performance when participating firms were in growing markets, which may suggest that participation allowed them to leverage the market expansion.

Policy Evaluation Methodology

The aim of our analysis was to evaluate the policy impact of the TAA for Firms on firms participating in the program. Typically, we do not observe the counterfactual state—how firms that participated would have performed had they not participated—and we do not observe outcomes associated with a control group of firms that did not participate. A control group would require identifying a group of firms that is as much like the firms in our sample as possible but did not participate in the program. However, selecting a group that was affected by import penetration similarly to the actual participant group would be quite difficult. Moreover, using a control group has weaknesses. For example, macroeconomic and other trends could affect the two sets of firms differently.

Given these data challenges, we measured the average effect of TAA for Firms participation on the firms in our sample. We made inferences from our regression estimates only for this sample. While this approach allowed us to focus on the firms that actually participated, it cannot determine whether these firms might have improved in the absence of the program. It is possible that participating firms have some unobservable or unmeasured characteristic—such as superior management—relative to firms that did not participate. In such circumstances, participation in the program might be incidental to improved performance.

Discussion of Empirical Model, Data, and Variables

Overall, our data set included 579 firms, or panels, and 2,711 observations. We used a panel of cross-sectional, time-series data that included values for variables across a number of years for a group of firms. To obtain data on performance of the various firms in the program, we sent a data collection instrument to TAA Centers, requesting data on firms that had completed at least one project in their business recovery plan in 2009, 2010, or 2011. Our data collection instrument requested data on sales and employment 2 years prior to entry into the program, in the certification year, and 2 to 3 years after entry into the program as well as other quantitative and qualitative data for each firm. The panel of data had unequal time periods, as firms in our sample entered and exited the program at different times. For example, two firms that completed a project in 2009 might have entered into the program in different years. In addition, the data were nonsequential in time and included gaps. For instance, EDA did not collect data on sales and employment for every year and explained that some of the data could not be obtained or was missing. Although some regression methods can accommodate unbalanced panels, others cannot, and this placed certain constraints on our regression methods. The time periods of the panels for each firm averaged 4 years, with a minimum of 2 years and a maximum of 6 years.

This is reportedly a common pattern in collection of data on countries, states, or firms and is more the norm than the exception. For example, data for some time periods in panels cannot be collected because of lack of resources or funding. See Baltagi, Badi H., Economic Analysis of Panel Data, 2nd ed. (West Sussex, England: John Wiley & Sons, Ltd., 2003).

And, we define the independent or control variables as: Emp = yearly employment for firm (proxy for size of firm)

Emp2 = yearly employment squared for firm Age = age of firm Age2 = age squared of firm Growth = yearly growth in industry value of shipments for each industry Growth_lg1 = yearly growth in industry value of shipments, lagged one year TAAF Policy = trade adjustment assistance indicator = 0 before certification and 1 thereafter PolGrwth = interactive variable: TAAF Policy * growth Nmulti = indicator variable for multiplant firm; 1 if multiplant firm, 0 otherwise Nprodmkt = categorical variable indicating which level of the product market the firm sells to such as raw, intermediate, and final/consumer Ngeomkt = categorical variable indicating which geographic market the firm sells to, such as local, regional, domestic, international, or some combination of these markets Npubpriv = indicator variable for whether the firm is public or private; public = 1, private = 0 TPR = industry-level import penetration ratio for each 5 and 6-digit NAICS code Unemp = yearly regional unemployment rate (BLS Census Region)

Year = year trend variable (1998 through 2011)

A summary of the panel data showing the variables used, including the means, standard deviations, minimums and maximums, is shown in table 5.

For our main dependent variable, we used the natural log of firm sales to proxy firm performance. We transformed the sales variable into natural logarithms, because the distribution of firm sales was highly skewed to smaller sales amounts. When transformed into natural logarithms, the sales data exhibited more of a normal distribution, making it more useful for regression analysis and reducing the effects of outliers.

Our policy variable is represented in the model as an indicator variable denoted by 0 prior to a firm’s certification by a TAA Center and 1 after certification. Using this variable, we tested whether the policy variable going from 0 to 1 brought about a statistically significant increase in the log of sales over time for these firms. When a firm changes from nonparticipation to participation, sales would change by some percent. We also created an interactive term—polgrwth—which is the policy variable times the growth variable. This allows detection of whether changes in market growth over the period, in combination with the TAA for Firms policy, had a further effect on firm sales.

For our independent variables, we used firm-specific financial and qualitative variables, which we transformed into indicator variables. Our firm-specific variables included employment and firm age. In the data collection instrument we sent to TAA Centers, we asked for qualitative data such as reach of the geographic market, level of the product market, whether the firm was public or private, and whether it was part of a multiplant firm. These variables represent market structure characteristics such as size, product differentiation, and economies of scale of the firm.

We also created variables that were market- or industry-specific, such as growth and lagged growth. For this, we used Census of Manufacturing, Annual Survey of Manufactures value of shipments data on a yearly basis from 1998 to 2010, at the five- and six-digit North American Industry Classification System (NAICS) level.

According to “Frequently Asked Questions” on the NAICS website at url: http://www.census.gov/eos/www/naics/faqs/faqs in answer to the question, “What is the NAICS structure and how many digits are in a NAICS code?” The NAICS is a two- through six-digit hierarchical classification system, offering five levels of detail. Each digit in the code is part of a series of progressively narrower categories, and the more digits in the code signify greater classification detail. The first two digits designate the economic sector, the third digit designates the subsector, the fourth digit designates the industry group, the fifth digit designates the NAICS industry, and the sixth digit designates the national industry. The five-digit NAICS code is the level at which there is comparability in code and definitions for most of the NAICS sectors across the three countries participating in NAICS (the United States, Canada, and Mexico). The six-digit level allows for the United States, Canada, and Mexico each to have country-specific detail. A complete and valid NAICS code contains six digits.

We also created an international trade variable at the six-digit NAICS industry level for import penetration, using International Trade Commission (ITC) import and export data from its online Tariff and Trade Data Web and the Census Annual Survey of Manufacturers’ value of shipments data. We calculated this variable as follows: Imports / (Value of Shipments + Imports – Exports). This ratio measures imports as a proportion of apparent domestic consumption and is often used in the trade literature as a proxy for import penetration. We found that trade data were missing in the ITC database for some of the unique NAICS industries represented in our firm sample from the TAA Centers. Specifically, although there are about 579 firms in our sample overall, when matched with the trade data the sample size reduces to about 472 firms. Including this variable yields a smaller data set and somewhat different regression results.

Results of Our Analysis

Table 6 below presents the regression analysis results, for three model specifications, with 1) the dependent variable as the log of firm sales; 2) the dependent variable as the log of firm sales including the import 3) the dependent variable as the log of productivity (sales divided by employment).

The table also shows the regression coefficients and Z-value of the three regression equations. We estimated the model specifications using the feasible generalized least squares (FGLS) estimator because of autocorrelation or correlation in the error terms over time in the data.

We used the FGLS model, xtgls in Stata, a panel data regression technique that can correct for serial correlation. We tested for the presence of autocorrelation using the xtserial command in Stata and found that we could not reject the presence of serial correlation in the data. Also, in this type of model, the autocorrelation coefficient may be different across panels, since the dependent variables are likely to be autocorrelated within a panel or firm. Therefore, this estimator allowed us to specify a model that corrected for panel-specific autocorrelation. The FGLS model structure can also accommodate unbalanced panel data with missing observations as we have in our sample of data and other non-time-varying variables of interest. Use of the technique improved model fit, and our primary variables of interest, including the TAA for Firms policy variable, were significant and had the expected signs.

FGLS estimators are appropriate when one or more of the assumptions of homoskedasticity and noncorrelation of regression errors fails.

Model 1, which uses log of sales as the dependent or outcome variable, without the import penetration ratio, used 2,572 observations and 572 panels in the regression. Model 2, which included the import penetration ratio explanatory variable, reduced the sample to 2,086 observations and 466 panels or firms. Model 3 included productivity as the outcome variable, defined as the log of firm sales divided by firm employment, along with the other control and macroeconomic variables in the other two specifications; it contained the full sample of observations. The coefficients and Z values for the TAA for Firms policy variable are consistent across the 3 model specifications. In addition, the Wald Chi- squared test for model significance showed that all model specifications were statistically significant overall.

The TAA for Firms policy indicator variable, which is 0 prior to participation in the TAA for Firms program and 1 thereafter, is positively and significantly related to log of firm sales in our model specifications at the 1 and 5 percent levels. This implies an average of a 5 percent to 6 percent increase in sales when the firm participates in the program, with all else constant. However, these model coefficients are relatively small (0.046 to 0.057 for the sales models) in comparison with other variables such as the multiplant firm variable, the public/private ownership of the firm, and the lagged growth variable. Further, the TAA for Firms policy variable is also positively related and statistically significant at the 5 percent level in our productivity specification. This result translates into about a 4 percent increase in the level of productivity, on average, for participation in the program for firms in our sample, all else remaining constant. The interactive variable—growth—with the TAA for Firms policy variable is also positively and significantly related to the log of firm sales in Model 1. The model suggests that participation in the TAA for Firms program had an effect on sales that was greater in industries that were experiencing growth than in those that were not. As Mueller and Rogers point out, there is more room for expansion by fringe firms in the market when there is greater growth in market demand.

Mueller, Willard F. and Richard T. Rogers, “Changes in Market Concentration of Manufacturing Industries, 1947-1977,” University of Wisconsin, Madison, and the U.S. Department of Agriculture, March 1983.

Appendix IV: Data That EDA Uses for Its Funding Allocation Formula and Share of Funding by Center

The Economic Development Administration’s (EDA) funding allocation formula includes five factors to allocate funding to each of the 11 TAA Centers, in addition to base funding divided equally across all of the centers. Table 7 shows the data that EDA used to measure each of these five factors to allocate $15.4 million among the centers for the 1-year 2011 cooperative agreement period. The table shows for each factor the data by center as well as each center’s share of the total. For example, the Great Lakes Center accounted for 3.8 percent of geographic size, 9.7 percent of the total number of firms, 11.7 percent of the total number of employees in the program’s approved business recovery plans, and 8.1 percent of the total number of firms that achieved expected results. Table 8 shows the resulting allocation to each of the centers for cooperative agreement year 2011.

Appendix V: Comments from the Department of Commerce

Appendix VI: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Kim Frankena (Assistant Director), Christina Bruff, David Dayton, Leah DeWolf, David Dornisch, Barbara El Osta, Etana Finkler, Bradley Hunt, Ernie Jackson, Erin Preston, Kelly Rubin, and Andrew Stavisky made key contributions to this report.