Tax Policy and Administration: Review of Studies of the Effectiveness of the Research Tax Credit
Published: 1996-05-21T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Pursuant to a congressional request, GAO reviewed eight studies of the research tax credit, focusing on the: (1) adequacy of the studies' data and methods to determine the amount of research spending stimulated per dollar of foregone tax revenue; and (2) other factors that determine the credit's value to society.
section_title: What GAO Found
paragraphs: GAO found that: (1) four studies supported the claim that, during the 1980s, the research credit stimulated research spending that exceeded its revenue cost, but the other four studies did not support the claim or were inconclusive; (2) all of the studies had significant data and methodological limitations that made it difficult to evaluate industry's true responsiveness to the research tax credit; (3) the studies did not use tax return data to determine the credit's incentive because the authors did not qualify for access to such data; (4) publicly available data were not a suitable substitute for the tax return data because public sources used different definitions of taxable income and research spending; (5) the studies' analytical methods, such as use of industry aggregates and failure to incorporate important tax code interactions, made their findings imprecise and uncertain; (6) there was little research on the latest design of the credit to determine its effect on incentives and costs; (7) the studies' evidence was not adequate to conclude that a dollar of research tax credit would stimulate a dollar of additional short-term research spending or about two dollars of additional long-term research spending; and (8) to measure the credit's true impact, the studies would need to assess the research's net benefit to society, resource costs of research, and administrative, compliance, and efficiency costs of funding the credit.

section_title: Letter
paragraphs: During a May 1995 hearing on the future of the research tax credit, you asked us to evaluate recent studies of the effectiveness of the research tax credit, particularly those cited in a report prepared by KPMG Peat Marwick, to determine whether the studies provide adequate evidence to conclude that each dollar taken of the tax credit stimulates at least one dollar of research spending in the short run, and about two dollars of research spending in the long run. This report responds to your request and evaluates the six studies cited in the KPMG Peat Marwick report as well as two other recent studies of the credit. Our objectives were to (1) evaluate the studies for the adequacy of the data and methods used to determine the amount of research spending stimulated per dollar of forgone tax revenue and (2) identify factors other than spending per dollar of revenue cost that determine the credit’s value to society.
subsections: section_title: Background
subsections: section_title: History of the Research Tax Credit
paragraphs: In 1981, Congress created the research tax credit to encourage business to do more research. The credit has never been a permanent part of the tax code. Since its enactment on a temporary basis in 1981, the credit has been extended six times and modified four times.
The research tax credit has always been incremental in nature. Taxpayers receive a credit only for qualified research spending that exceeds a base amount. Beginning in 1981, taxpayers could reduce their tax liability by 25 percent of qualified research that exceeded a base amount that was equal to the average research expenditure of the 3 previous years or a base amount that was equal to 50 percent of the current year’s expenditures, whichever was greater.
The Tax Reform Act of 1986 modified the credit by reducing the rate to 20 percent of qualified spending above the base amount and more narrowly defining qualified expenditures. The credit was changed again in 1988 to require that taxpayers reduce their deductions for research expenditures by an amount equal to 50 percent of the credit they claim. In 1989, this amount was increased to 100 percent of the credit they claim.
The Omnibus Budget Reconciliation Act of 1989 changed the method for calculating the base amount. The base calculated as the average expenditure of the 3 previous years was replaced by a base amount equal to the ratio of total qualified research expenses to total gross receipts for 1984 through 1988, multiplied by the average amount of taxpayer’s gross receipts for the preceding 4 years. This base change removed the link between increases in current spending and future base amounts that had reduced the incentive to undertake additional research spending under the prior method for calculating the base.
section_title: Early Studies of the Credit’s Effectiveness
paragraphs: The evaluation of the effectiveness of the credit requires first estimating the additional research spending stimulated by the credit. Ideally, this additional spending should then be evaluated according to the net benefit it produces for society. However, this net social benefit is difficult to determine because it depends on how the research of some companies affects the costs and products of other companies. Some researchers who have studied the credit have instead calculated a “bang-per-buck” ratio, the amount of spending stimulated per dollar of revenue cost. Once a decision has been made to provide some form of credit, this ratio is a relevant criterion for assessing alternative designs.
Most early studies of the research tax credit found that, although the credit may have stimulated some additional research spending, the effect on spending was relatively small. For example, Edwin Mansfield in his 1986 study asked a random sample of corporate officials to assess the effect of the credit on research spending and estimated from their responses that the additional spending induced by the credit equaled about one-third of the revenue cost. Robert Eisner, et al., in 1984 compared the growth of research spending that qualified for the credit and spending that did not qualify in 1981 and 1982, and found no positive impact of the credit on the growth of research spending.
Other early studies relied on estimates of the responsiveness of research spending to reductions in its price to arrive at similar conclusions. Because the credit is effectively a reduction in the price of research, the greater the responsiveness of research spending to price reductions, the more additional spending the credit is likely to stimulate. Economists measure the responsiveness in terms of the “price elasticity” of spending, which shows the percentage increase in spending that would result from a 1-percent reduction in the after-tax price of research and development (R&D).
In 1989, we reported that the best available evidence indicated that research spending is not very responsive to price reductions. Most estimates of the price elasticity of spending fell in the range of –0.2 to –0.5, implying that a 1-percent reduction in the price of research would lead to between a 0.2 percent and 0.5 percent increase in spending.
In our 1989 report, we used Internal Revenue Service (IRS) data to estimate that between 1981 and 1985, the credit provided companies with a benefit of 3 to 5 cents per dollar of additional spending. This benefit to companies is equivalent to a reduction in the price of research. Combining these price reductions with the range of elasticity estimates, we estimated that the credit stimulated between $1 billion and $2.5 billion of additional research spending between 1981 and 1985 at a cost of $7 billion in tax revenue. Thus, we estimated that each dollar of taxes forgone stimulated between 15 and 36 cents of research spending.
section_title: Reviews of Recent Studies of the Credit’s Effectiveness
paragraphs: Reports on the research tax credit by KPMG Peat Marwick and by the Office of Technology Assessment (OTA) include reviews of studies of the credit’s effectiveness that were issued since our 1989 report. The KPMG Peat Marwick report concludes that the studies provide evidence that the spending stimulated by the credit equals or exceeds its revenue cost. Specifically, the report concludes that the recent studies show that one dollar of credit stimulates about one dollar of R&D spending in the short run, and as much as two dollars in the long run. According to the KPMG Peat Marwick report, the recent studies KPMG Peat Marwick reviewed provide better estimates of the effectiveness of the credit than earlier studies because they analyze longer data series and because they use what it considered to be better methodologies for analyzing the effect of the credit. The OTA report reviewed the same recent studies as KPMG Peat Marwick and observed that the available literature generally reports that the credit stimulates about one dollar of additional spending per dollar of revenue cost. However, OTA pointed out that the studies contain data and methodological uncertainties. For our review, we evaluated the studies cited in these two reports as well as other studies not included in either report. We also addressed some methodological issues that were not addressed in these reports and provided a more detailed evaluation of each study.
section_title: Objectives, Scope, and Methodology
paragraphs: Our first objective was to evaluate recent studies of the research tax credit for the adequacy of the data and methods used to determine the amount of research spending stimulated per dollar of revenue cost. In particular, we were to determine if recent studies provided adequate evidence to conclude that each dollar of tax credit stimulates at least one dollar of research spending in the short run and, over the long run, stimulates about two dollars of research spending. Our second objective was to identify the factors other than spending per dollar of revenue cost that determine the credit’s value to society.
To meet our first objective, we reviewed the six studies cited by the KPMG Peat Marwick report and two studies that the report did not cite that we identified from our review of the literature on the credit and from our interviews with authors of research tax credit studies. In general, these recent studies were published since our 1989 report, although one study cited by KPMG Peat Marwick was published in 1987. The studies are listed in appendix I.
We used standard statistical and economic principles in our review and evaluation of the studies of the research tax credit. We relied upon internal economists to carry out this evaluation. In our evaluation, we considered such factors as the adequacy of the data used to estimate the effect of the credit, the adequacy of the variables used to measure the incentive provided by the credit, and the sensitivity of the estimates to assumptions about taxpayer behavior.
We also interviewed the authors of the studies of the research tax credit and requested comments on a draft of our evaluation of their studies. We received comments from the authors of six of the eight studies that we reviewed. All agreed that our report accurately summarized their studies. However, not all agreed with the importance of the data and methodological limitations that we identified in their work. A summary of their comments appears on pages 12 and 13. We also requested comments on a draft of our report from the authors of the KPMG Peat Marwick report. They stated that they appreciated the opportunity to comment on our report but that after reviewing our report, they had no comments to submit.
To meet the second objective, we reviewed academic articles and government studies about the determinants of the social benefits of research spending. We also reviewed studies that describe the difficulties encountered when attempting to measure the full social costs and benefits of research.
We did our work in Washington, D.C., from December 1995 through January 1996 in accordance with generally accepted government auditing standards.
section_title: Recent Studies Provide Mixed Evidence of the Credit’s Effectiveness
paragraphs: The recent studies that we reviewed provided mixed evidence on the amount of spending stimulated per dollar of revenue cost. Of the eight studies we reviewed, three supported the claim that one dollar of credit stimulated about two dollars of additional research spending. Another study, which did not directly evaluate the research tax credit, reported estimates of the responsiveness of research spending to other tax incentives. These estimates appear to be consistent with the claim that the credit stimulates spending that exceeds its revenue cost.
However, two studies reported that the credit stimulated spending that was less than its revenue cost, and another two of the studies reported estimates of additional spending that do not appear to support the claim that spending exceeded revenue cost. One of these latter studies does not compare additional research spending to revenue cost but does report an estimate of additional spending that is likely to be less than the revenue cost. The other study reported that additional spending exceeded revenue cost through 1985 but reported estimates of additional spending that were likely to be less than the revenue cost after 1985.
Most of the recent studies used more sophisticated methods than prior studies when analyzing the effectiveness of the credit. For example, the studies improved on prior studies by using methods that attempt to distinguish the credit’s effect from other influences on research spending like the potential size of the market for the product of the research. However, the studies have the following data and methodological limitations.
subsections: section_title: Limitations of the Data Used by the Studies
paragraphs: The most appropriate data for assessing the effect of the credit are tax return data. These confidential tax return data were not available to the authors of the studies. Instead, they used publicly available data sources, chiefly the COMPUSTAT data service, which do not accurately reflect the incentive provided by the credit. This incentive depends on a company’s ability to earn credits by having qualified research spending that exceeds the base amount and on a company’s ability to claim its credits by having sufficient taxable income. We concluded from our own comparison of tax return data with COMPUSTAT data and from studies by other researchers that differences in the measurement of research spending and taxable income make COMPUSTAT an unreliable proxy for tax return data when analyzing the credit. Because studies that use the public data cannot accurately determine the credit’s incentive, they may not accurately measure the amount of spending stimulated by the credit.
section_title: Limitations of the Studies’ Methods for Measuring Tax Incentives
paragraphs: Three studies that analyzed the credit at the industry level may not accurately measure the credit’s incentive. Analysis at the industry level of aggregation does not reflect the different incentives the companies face and their different responses to these incentives. Industries include firms that earn no credit because their spending is less than the base amount or claim no credit because they have no tax liability. An analysis at the industry level that assigns the same incentive to all these firms would not capture these differences and is not likely to produce very precise measures of the credit’s effect on research spending.
The eight studies all used measures of the tax incentive that did not incorporate important interactions with other features of the tax code. For example, studies that measured the tax incentives by reductions in the cost of research and development due to tax policy changes did not include all the research and development provisions of the tax code. In addition to the credit, the cost of R&D depends on other tax code provisions like those governing the allocation of research expenses between foreign sources and the United States. The studies included some of these provisions but not others. Including all relevant provisions of the code may change the estimates of the research credit’s effectiveness.
The estimates in several of the studies were highly sensitive to assumptions made about the data and taxpayer behavior. For example, one study’s estimate of the responsiveness of spending to tax incentives was reduced by half when more firms were included in the sample studied or the assumptions were changed on how taxpayers allocate research and development expenses between domestic and foreign sources. Other studies that differed in terms of how they measured the tax incentive produced significantly different estimates of the spending stimulated by the credit. This sensitivity to the assumptions made by the authors leads us to conclude that much uncertainty remains about the effect of the credit on research spending.
section_title: The Amount of Spending Stimulated Per Dollar of Revenue Cost Depends on the Design of the Credit
paragraphs: The estimates presented in the most recent studies do not provide all the information needed to evaluate the effectiveness of the latest version of the credit. The amount of spending stimulated per dollar of revenue cost depends on how the design of the credit affects the incentive to increase research spending and on how the design affects the revenue cost. Only one of the recent studies estimated the effectiveness of the credit for years after its redesign in 1989, and the author of that study is not confident of her results for the post-1989 period.
Some reviewers have implied that the recent studies’ estimates of the responsiveness of research spending to price reductions—the price elasticity of spending—are equivalent to the amount of research spending stimulated by the credit per dollar of revenue cost. They said that using an empirical estimate that a 1-percent reduction in the price of R&D will lead to a 1-percent increase in research spending implies that one dollar of credit will lead to one dollar of additional spending. However, these may not be equivalent estimates because the amount of research spending stimulated by the credit per dollar of revenue depends on the design of the credit as well as the responsiveness of spending to price reductions.
For example, the credit’s effect on spending and revenue cost will depend on whether it is designed as a flat credit, which applies to total research spending, or as an incremental credit, which applies only to spending that exceeds a base amount. For the same responsiveness of spending to price reductions, a flat credit with a 10 percent rate should stimulate roughly the same amount of spending as an incremental credit with the same rate because both credits provide the same 10 percent effective reduction in the price of research. However, the flat credit would allow a company to earn a credit equal to 10 percent of its total qualified research spending, while the incremental credit would give the company a credit equal only to 10 percent of the difference between its current qualified spending and some base spending amount. Consequently, the 10 percent flat credit would have a higher revenue cost and, therefore, a lower bang-per-buck than the 10 percent incremental credit.
Incremental credits that differ from one another in terms of how base spending is defined can also differ substantially in terms of how much spending they stimulate per dollar of revenue cost. The bang-per-buck of the current incremental credit may be significantly different from that of the credit that existed prior to 1990. As we reported in our May 1995 testimony, the redesign of the credit in 1989 should have increased the size of the incentive provided per dollar of revenue cost. However, as we also reported in our testimony, there is evidence that the incentive provided by the redesigned credit had eroded over time and that the revenue cost of the additional spending stimulated by the credit had increased.
section_title: Factors Determining the Research Tax Credit’s Value to Society
paragraphs: The value of the research tax credit to society cannot be determined simply by comparing the amount of research spending stimulated by the credit versus the credit’s revenue cost. To fully evaluate the credit’s effect, one would have to (1) estimate the total benefits gained by society from the research stimulated by the credit; (2) estimate the resource costs of doing the research; (3) estimate the administration, compliance, and efficiency costs to society resulting from the collection of taxes (or the borrowing of money) required to fund the credit; and (4) compare the benefits to the costs. Simply knowing how much additional research spending the credit stimulates does not tell you the value of that research to society. Similarly, the amount of revenue needed to fund the credit does not tell you the total cost to society of the credit.
There is a general consensus among economists that research is one of the areas where some government intervention in the marketplace may improve economic efficiency. From society’s point of view, individual companies may invest too little in research if the return on their investment is less than the full benefit that society derives from the research. If the research leads to new products, reduces costs or increases productivity for other companies and consumers throughout the economy, the benefits to society may exceed the return on investment of the companies that conduct the research. Therefore, companies may not do as much research as society finds desirable, and government policy to encourage research may be viewed as appropriate.
However, as the Joint Committee on Taxation and OTA have noted, it is also possible to decrease economic efficiency by encouraging too much spending on research. Because not all research generates social benefits that exceed the returns to companies conducting the research, encouraging more research may not be economically efficient. It would be very difficult to determine, given the difficulty of measuring the social benefit, whether the research tax credit increases or decreases economic efficiency. No one that we are aware of, including the authors cited by KPMG Peat Marwick, has undertaken a study that could answer that question conclusively.
section_title: Authors’ Comments on Our Review of Their Studies
paragraphs: As previously discussed, we requested comments from the authors of the KPMG Peat Marwick report. After reviewing a draft of our report, they stated that they had no comments to submit. We also requested that the authors of the eight studies of the research tax credit that we reviewed provide comments on our evaluation of their studies. The following summarizes the comments of the six authors who responded to our request.
All of the authors we interviewed agreed that the publicly available data contain measurement errors that may affect their estimates of the credit’s effectiveness. However, two of the authors said that they believed that their estimates would not change significantly if tax return data were used. They said that either the data problem was minor or that statistical methods used to correct the measurement error reasonably addressed the problem. Two authors also said that they believed that their elasticity estimates would not change significantly but noted that predicting what would happen to the estimates when better data are used is difficult. Two authors agreed with our assessment of the importance of the potential inaccuracies from using COMPUSTAT data.
As explained more fully in appendix I, we have concluded that COMPUSTAT data are not a suitable proxy for tax return data when analyzing the credit. Although the authors agree that COMPUSTAT data are not the best data, they disagree among themselves about the importance of this issue. We acknowledge that statistical methods can be used to help address this issue of measurement error, but the success of these methods is difficult to assess. We conclude that, because the most appropriate data were not used in these studies, uncertainty remains about the responsiveness of spending to the credit.
The methodological limitations that we identified were not addressed in the comments of all the authors because they were not relevant to every study. The authors who did comment disagreed about the importance of the methodological limitations. One author who addressed the importance of correctly incorporating the features of the tax code said she believed that some of the studies’ estimates of the effect of the credit were overestimated because the method of estimation excluded tax preferences available for investments other than research. Another author commented that the sensitivity of the estimates to assumptions about taxpayer expectations accounted for the difference in estimates across the studies. However, two authors who agreed with our identification of the methodological limitations in their work did not believe that the limitations had a significant effect on their estimates. Finally, the authors who commented agreed that analyzing the credit at the firm level rather than at the industry level produces more accurate estimates. However, one author said that he did not believe that his industry level estimates would change significantly if they were based on analysis at the firm level.
As explained in appendix I, we found that estimates reported in the studies varied significantly when authors employed different assumptions about the data and taxpayer behavior. This sensitivity of the estimates to authors’ assumptions leads us to conclude that much uncertainty remains about the effect of the credit on research spending.
We are sending copies of this report to pertinent congressional committees, the Secretary of the Treasury, KPMG Peat Marwick, the individual authors, and other interested parties. Copies will be made available to others upon request.
The major contributors to this report are listed in appendix II. If you have any questions, please call me on (202) 512-9044.
section_title: Evaluation of Studies of the Effectiveness of the Research Tax Credit
paragraphs: We classified the studies of the effectiveness of the research tax credit according to the level of aggregation at which the data are analyzed and the method used to measure the incentive provided by the credit. The studies analyze the credit using firm level data or using data aggregated to the industry level. The incentive provided by the credit is measured by a categorical or “dummy” variable, or by a variable measuring the “tax price” of research and development (R&D). The categorical variable measures the change in R&D spending due to the presence or absence of the tax credit or to the ability of firms to use the credit, while the tax price variable measures the change in spending due to the effect of tax policy on the cost of R&D.
We reviewed the six studies cited by KPMG Peat Marwick in their report.We also reviewed two recent studies of the credit’s effectiveness that were not cited by KPMG Peat Marwick. The following summarizes the studies and presents our evaluation of them.
subsections: section_title: Descriptions of the Studies
subsections: section_title: Industry Level Studies
paragraphs: Martin Neil Baily and Robert Z. Lawrence use National Science Foundation (NSF) data to examine the effect of the credit for 1981 through 1985 in their 1987 study, and for 1981 through 1989 in their 1992 study. The 1987 study analyses the credit using a dummy variable that indicates the years in which the credit was in effect, while the 1992 study uses a variable that reflects changes in the credit’s incentive due to changes in the tax law. Both studies produce essentially the same finding: the percentage increase in R&D spending in response to each percentage decrease in the price of R&D—the price elasticity of R&D—is approximately equal to one. Using this elasticity, Baily and Lawrence estimate that the credit generated about two dollars of R&D for each dollar of tax revenue forgone.
Theofanis P. Mamuneas and M. Ishaq Nadiri use industry level data for 1956 through 1988, chiefly drawn from the Bureau of Labor Statistics and NSF. Their method is to construct a rental price variable for R&D capital that reflects the research tax credit and the provisions for the immediate expensing of research expenditures. To construct this variable, the authors acknowledge that they assume that the firms in their industries have sufficient tax liability to claim the credit, that their spending exceeds the base amount, and that spending is less than twice the base amount. Their estimates of price elasticities range from –1.0 for the three aggregate industries of textiles and apparel; lumber, wood products, and furniture; and other manufacturing to –0.94 for scientific instruments. On the basis of these elasticities, they calculate that the average additional research spending stimulated per dollar of revenue cost was about 95 cents for the period 1981 to 1988.
section_title: Firm Level Studies
subsections: section_title: Studies Using Tax Price Variables
paragraphs: James R. Hines’ study uses firm level data from COMPUSTAT for 1984 through 1989. His method is to construct a tax price variable that measures how the costs of R&D are affected by the rules for allocating R&D expenses between U.S. and foreign sources under section 1.861-8 of U.S. Treasury regulations. His tax price does not include the research tax credit or other R&D related features of the tax code. Hines’ preferred estimates of the R&D price elasticity range from –1.2 to –1.6. However, when he increases his sample size to include firms previously excluded due to merger activity, these elasticity estimates drop to a range of –0.5 to –0.6. Also, the elasticities decrease to –0.5 to –0.9 when Hines changes his assumptions about how firms allocate their research expenses. Hines does not apply these elasticities to the credit or calculate how much spending is induced by the credit.
Bronwyn H. Hall uses firm level data from COMPUSTAT for 1977 through 1991. Her tax price variable measures how the research tax credit and expensing provisions affect the cost of R&D. Hall estimates a short-run price elasticity of R&D of –1.5 and a long-run price elasticity of –2.7. However, she advises that the long run elasticity be viewed with caution, as it is likely to be “quite imprecise.” Hall estimates that the additional spending induced by the credit in the short run was $2 billion per year, while the tax revenue cost was about $1 billion per year.
section_title: Studies Using Dummy Variables
paragraphs: Philip Berger’s study uses firm level data from COMPUSTAT for 1975 through 1989. He measures the effect of the credit using a dummy variable that indicates the years in which a firm is able to use the credit, i.e., the firm has a positive tax liability in the current or preceding 3 years. Berger uses the results of this analysis to estimate that the credit induced $2.70 billion of additional spending per year from 1982 through 1985. He compares this yearly increase to a yearly revenue cost of $1.55 billion to conclude that additional spending per dollar of forgone revenue was $1.74 during 1982 through 1985. Although Berger does not calculate the amount of spending per dollar of forgone revenue for years after 1985, his study shows that the credit was less effective in later years.
C. W. Swenson’s study uses firm level data from COMPUSTAT for 1975 through 1988. He also uses a dummy variable that indicates the years in which a firm is able to use the credit. However, the ability to use the credit in his study depends not only on current tax status but also on future tax status and the firms’ planned R&D spending. Swenson estimates that total additional spending induced by the credit was $2.08 billion during 1981 through 1985. Swenson does not compare this estimate to the revenue cost.
Janet W. Tillinger’s study uses firm level data drawn chiefly from COMPUSTAT for 1980 through 1985. She measures the effect of the credit using a dummy variable that indicates the years in which firms have research spending that exceeds the base amount. Tillinger uses the results of this analysis to estimate that the credit induced about 19 cents of increased spending per dollar of forgone revenue for 1981 through 1985, which she notes is at the lower end of the estimates from our 1989 study. Tillinger also finds that the effectiveness of the credit varies by the type of firm. When the firms are classified according to the opportunity costs of alternatives to R&D investment like the payment of dividends, she finds that the additional spending ranges from 8 cents to 42 cents per dollar of forgone revenue.
The studies reviewed above provide mixed evidence for claims about the amount of spending induced by the credit per dollar of forgone revenue. Of the six studies cited by KPMG Peat Marwick, three studies (the two by Baily and Lawrence, and Hall’s study) support the claim that each dollar of tax revenue stimulated about two dollars of additional research spending. Hines’ study reports a price elasticity of research spending that, if applied to the research tax credit, is likely to be consistent with the finding that additional spending exceeds the revenue cost.
Two studies cited by KPMG Peat Marwick, however, may not support the claim that induced research spending exceeds the revenue cost of the credit. Swenson’s study estimates that the credit induced additional spending of $2.08 billion from 1981 through 1985. He notes that his estimate is “comparable to . . . GAO estimates of $1 billion to $2.9 billion for the same period.” Swenson states that he does not calculate a bang-per-buck measure because he does not have access to the taxpayer data necessary to make this calculation. However, Swenson states that his estimate of additional spending is not likely to support the claim that the spending stimulated by the credit exceeded its revenue cost. Berger’s study estimates that additional spending exceeded revenue cost in the period 1982 through 1985, but the study may not support this claim in the years after 1985. Berger does not calculate a bang-per-buck measure for years after 1985. However, his study does show that the credit was less effective in these years and that the credit was not a statistically significant determinant of R&D spending in the years after 1986.
The two studies that were not cited by KPMG Peat Marwick do not support the claim that induced spending exceeded the revenue cost of the credit. The Mamuneas and Nadiri study estimates that the credit stimulated additional spending that was slightly less than the revenue cost during 1981 through 1988, while the Tillinger study estimates that additional spending was significantly less than revenue cost during 1981 through 1985.
section_title: Our Evaluation of the Studies
paragraphs: Most of the studies we reviewed use more sophisticated statistical methods and more years of data than prior studies. For example, most of the recent studies use methods that attempt to distinguish the credit from other factors that influence research spending like market size and the availability of investment funds. Some studies also include the influence of taxpayers’ expectations about factors like the future tax status of firms when determining the effect of the credit on current spending. Nevertheless, despite these advantages over prior studies, these studies have data and methodological limitations that are significant enough to lead us to conclude that much uncertainty remains about the true responsiveness of research spending to tax incentives.
None of the studies use the best data for assessing the effect of the credit. They all use publicly available COMPUSTAT or NSF data, which are not the most appropriate data for this purpose. The incentive provided by the credit depends on companies’ ability to earn credits by having qualified research spending that exceeds the base amount, and to claim credits by having tax liabilities. Information on qualified research spending and tax liabilities can be most accurately determined from confidential IRS data. The publicly available data will not be as accurate because they use definitions of research spending and tax liabilities that are different from IRS. These tax return data were unavailable to these researchers. In her study, Hall recognizes the limitations of publicly available data and attempts to correct the errors in her measurements. However, it is difficult to determine how successful her efforts are without repeating her analysis using the tax return data. In any case, the estimates of all the studies that we reviewed would be more reliable if they were based on IRS data.
The tax price variables and the dummy variables used in the studies to capture the incentive provided by the credit depend on companies’ ability to earn credits and claim them against their tax liabilities. COMPUSTAT and NSF data do not accurately reflect credits earned and claimed.
The ability to earn credits depends on the relationship of qualified research spending to the base amount. COMPUSTAT and NSF data do not accurately reflect this relationship because both data sources include spending that does not qualify for the credit. Most notably, spending reported by COMPUSTAT includes spending overseas that would not be qualified research spending. In our 1989 report, we compared COMPUSTAT data with tax return data and concluded that COMPUSTAT data are not a suitable proxy for tax return data when analyzing the credit. For example, when we compared the growth rate of COMPUSTAT research spending with qualified research spending for a sample of firms contained in both the COMPUSTAT database and IRS files, we found that the rates varied considerably over the period 1981 through 1985. Qualified spending grew 1.46 times as fast as COMPUSTAT spending in the 1980 to 1981 period, but only 0.72 times as fast in the 1983 to 1984 period. The relationship between spending and the base using COMPUSTAT may not accurately reflect the relationship using tax data, and, therefore, both tax price variables and dummy variables are likely to be inaccurate.
The ability to claim credits depends on the tax status of the firms. COMPUSTAT contains information on taxable income and loss carryforwards, but studies have shown that COMPUSTAT does not always accurately or consistently reflect IRS data. Furthermore, COMPUSTAT data contain no information on the general business credit, which limits the ability of companies to claim the credit. Again, because both the tax price variables and the dummy variables depend on the ability of firms to claim the credit, we conclude that they will be measured inaccurately when based on COMPUSTAT data.
The reliability of the Baily and Lawrence studies and the Mamuneas and Nadiri study is also limited by the level of aggregation at which the data are analyzed. Their analyses of the credit at the industry level are unlikely to produce very precise measures of the credit’s effect. Their analyses do not reflect the different incentives that companies face and the different responses to these incentives. Industries will include firms that earn no credit because their spending is less than the base, firms that cannot claim the credit because they have no tax liability, and firms subject to the 50-percent base limitation. A measure that assigns the same incentive to all these firms will not capture these differences and is not likely to yield precise or reliable estimates of the credit’s effect.
The reliability of the studies that we reviewed is also limited by the methods used to measure the incentive provided by the credit. The studies use measures of the tax incentive that do not incorporate important interactions of the research tax credit with other features of the tax code. For example, Hines studies the effect of the section 1.861-8 allocation rules on research spending but does not analyze the effect of other features of the tax code such as the research tax credit. Hall, on the other hand, analyzes the research tax credit but does not incorporate the section 1.861-8 allocation rules in her study. Hall believes that including the rules in her analysis would not make “an enormous difference” because the firms subject to the rules probably represent only a small part of her sample. However, she does say that including the rules would make her estimates more precise. Hines states that it is “difficult to know for sure” the effect on his estimates of including interactions with other features of the code.
The estimates in some of these studies are also uncertain because they are sensitive to assumptions made about the data and taxpayer behavior. Hines’ estimate of the effect of tax policy on spending is reduced by half when he includes more firms in his sample or changes his assumptions about how companies allocate R&D expenses. Hall notes that estimation at the firm level involving investments like R&D is difficult and sensitive to assumptions made when specifying the models. This sensitivity of the results is also illustrated by the three studies using dummy variables where differences in the approach to modeling taxpayer behavior and measuring the effect of the credit yield very different estimates. Although some of the studies attempt to measure the degree of this sensitivity and correct for it, the success of these efforts is difficult to assess.
The authors of these studies themselves, in many cases, advise that their results be used with caution and recognize that their estimates would benefit from further research. For example, when describing her estimates of the spending induced by the credit, Hall states that “it needs to be kept firmly in mind that my tax estimates are not likely to be as good as those constructed using IRS data.” She also mentions, in the 1992 version of her paper, that her analysis “needs more investigation for robustness over time and industry.” When discussing the limitations imposed by not including interactions with other aspects of the tax code, Hines notes that his results should be used with caution because of these “restrictive assumptions built into the estimated R&D responses to tax changes.”
The current version of the credit has not been studied extensively, and little is known about the actual incentives provided by the current credit. The research tax credit was fundamentally restructured in 1989. Hall’s study, which spans the years 1977 through 1991, is the only study we reviewed that covers any tax years after the credit was changed. However, her data contain only 2 years—1990 and 1991—under the revised credit structure. Hall notes that her estimate of additional spending for these years amounts to about 10 percent of the total R&D and that this “amount is almost too large to be credible . . . and deserves further investigation as more data become available.” She indicated that her estimates of additional spending may be less reliable because she did not have data on the tax status of firms after 1991 that were needed to measure the incentive provided by the revised credit.
section_title: Major Contributors to This Report
subsections: section_title: General Government Division, Washington, D.C.
paragraphs: James Wozny, Assistant Director, Tax Policy and Administration Issues Kevin Daly, Senior Economist Anthony Assia, Senior Evaluator The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.
U.S. General Accounting Office P.O. Box 6015 Gaithersburg, MD 20884-6015 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000  or by using fax number (301) 258-4066, or TDD (301) 413-0006.
Each day, GAO issues a list of newly available reports and testimony.  To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone.  A recorded menu will provide information on how to obtain these lists.