text: Background This section provides an overview of patenting in the United States, patent infringement litigation, and administrative proceedings for patent validity challenges. It also includes a brief history of court decisions that clarified eligibility requirements for the Patent Trial and Appeal Board’s CBM program. See “Related GAO Products” at the end of this report for a list of our prior work related to patents and intellectual property. Patenting in the United States In the United States, patents may be granted by USPTO for any new and useful process or machine, or any new and useful improvement on an existing process or machine, but there are some exceptions. Laws of nature, physical phenomena, and abstract ideas are not patentable. The U.S. Supreme Court and the U.S. Court of Appeals for the Federal Circuit have refined the boundaries of these exceptions over time, allowing some subject matter that was previously not patentable to become so. For example, U.S. Supreme Court decisions in the 1970s found mathematical formulas used by computers (i.e., software) were like laws of nature and therefore not patentable subject matter. However, a 1981 Supreme Court decision overturned USPTO’s denial of a patent application for a mathematical formula and a programmed digital computer because, as a process, the claimed invention was patentable subject matter. Similarly, business methods were widely considered unpatentable subject matter until 1998, when the U.S. Court of Appeals for the Federal Circuit ruled in the State Street Bank decision that they were patentable. In 2014, however, the Supreme Court effectively limited the patentability of some business methods by ruling in Alice Corp. Pty. Ltd. v. CLS Bank Int’l that using a generic computer to implement an abstract idea is not patentable. Traditionally, economic theory has held that intellectual property rights, such as those conferred by patents, can help encourage innovation and stimulate economic growth. Exclusive rights provided by patents, for example, can help patent owners recoup investments in technology and earn greater profits than if their patented technologies could be freely imitated. Moreover, to the extent that intellectual property rights encourage specialization, innovators may be more productive than they would be in the absence of patent laws. Because of complex trade-offs, however, some economists hold a more nuanced view of the potential for patents to promote innovation and increase productivity. By increasing the cost of using technologies, for example, patents may discourage not only diffusion of these technologies but also cumulative innovation that uses such technologies to develop new technologies. In addition, attempts to quantify the effect of patents on economic growth often fail to account for the creation of useful knowledge outside the patent system. Furthermore, to the extent that innovation occurs in the absence of patent laws, the need for patents can vary across industries or over time. Some researchers have suggested that some patents are currently limiting innovation, especially in areas such as software and computer technologies that overlap with business methods. USPTO receives hundreds of thousands of applications each year from inventors seeking patents to protect their work. According to USPTO data, applications for patents have increased in recent years, and the share of patents granted for business methods has significantly increased over the past 2 decades (see fig. 1). In calendar year 2014, patents related to business methods accounted for more than 28 percent of all issued patents. A patent’s claims define the legal boundaries of the invention, often in complex technical language. A patent application can be written to define an invention broadly or narrowly. Patent applicants often prefer broader claims because their competitors are less able to avoid infringement by making only small changes to their patented invention, as we reported in June 2016. Before issuing a patent, USPTO patent examiners determine whether claimed inventions in the application meet requirements for patentable subject matter, novelty, non-obviousness, and clarity—the four patentability grounds that are established by statute. Patent examiners assess whether the claimed invention consists of patentable subject matter and also ensure that the claims are described clearly enough to enable a person skilled in the art to make the claimed invention. In addition, examiners determine whether a patent application’s claimed invention is novel and non-obvious by comparing the application’s content to “prior art”— existing patents and patent applications both in the United States and abroad, as well as non-patent literature such as scientific articles. In February 2015, USPTO launched an Enhanced Patent Quality Initiative, which included several proposals designed to improve the quality of patent examination and issued patents. However, we found in June 2016 that USPTO faced challenges in issuing patents in accordance with standards. For example, we found that a majority of examiners (67 percent) said they have somewhat or much less time than needed to complete an examination, given a typical workload, and many examiners felt a time pressure that reduced their ability to conduct thorough searches. Examiners also said that it was difficult to issue patents that met the statutory requirements because of the limited availability of and access to non-patent prior art such as offers for sale and public use. Examiners said another limitation is their being responsible for examinations in subject areas in which they do not have adequate technical knowledge. We made seven recommendations to USPTO aimed at improving patent quality, clarity, and prior art search. USPTO agreed with the recommendations and is working to address them. Patent Infringement Litigation Patent owners can bring infringement lawsuits against anyone who uses, makes, sells, offers to sell, or imports the patented invention without authorization. Only a small percentage of patents in force are ever litigated, but some scholars believe that low-quality patents can make such litigation not only more complex and expensive but also more frequent. During an infringement case, the accused infringer may seek to have the lawsuit dismissed by showing the patent is invalid. When the courts rule on validity, they generally invalidate almost half of the patents, according to academic research. Exactly what a patent covers and whether another product infringes the patent’s claims are rarely easy questions to resolve in litigation, and defending a patent infringement lawsuit in district court can take years and cost millions of dollars, not including damages if infringement is found. Whatever the outcome, costly litigation can leave defendants with fewer resources for innovation. Consequently, patent infringement defendants often find it in their best interest to settle lawsuits quickly, as we reported in August 2013. Administrative Proceedings for Challenging Patent Validity before the Patent Trial and Appeal Board The AIA in 2011 created the Patent Trial and Appeal Board and stated any references in federal law to USPTO’s then-existing Board of Patent Appeals and Interferences be deemed to refer to the new board. By statute, the Patent Trial and Appeal Board consists of the USPTO Director, Deputy Director, Commissioner for Patents, Commissioner for Trademarks, and administrative patent judges. In practice, to issue decisions in the matters that come before it, the board involves more than 300 people serving in many positions, according to the board. The board is led by the Chief Judge and Deputy Chief Judge, who, along with other members of senior management, meet regularly to discuss operational and procedural matters of importance to the board’s overall mission, according to the board. The AIA created three new administrative proceedings for the board to administer, each with different statutory rules (see table 1). Two proceedings were made permanent: Post-grant review provides a 9-month opportunity following the issuance of a patent during which a third party can file a petition to challenge a patent’s validity on any of the four statutory grounds: subject matter eligibility, novelty, non-obviousness, and clarity. Inter partes review is available to third parties for the life of the patent, but on a limited set of grounds (non-novelty or obviousness), and on a limited set of acceptable prior art (previously issued patents and printed publications). The third proceeding—the CBM program—was included in the act as a temporary proceeding that can be used to challenge a patent at any point in its life, as allowable under the inter partes review program. However, under the CBM program, only a party (e.g., a company or an individual) that is sued or charged in an infringement suit can petition. Such petitioners can challenge a patent’s validity on any of the four statutory grounds without the limits on prior art in inter partes review. Additionally, rules about which arguments parties are officially barred from being raised again in later legal actions (called estoppel provisions) are less restrictive under the CBM program than for the other two board proceedings. However, the body of patents that qualify for review under the CBM program is limited to those that claim a non-technological method involved in the practice, administration, or management of a financial service or product. A patent is “technological” if it claims a technological feature that solves a technical problem using a technical solution. Many software and business method patents issued in the wake of State Street Bank describe implementing an abstract idea on a generic computer. Since the Supreme Court’s 2014 decision in Alice, which closely aligns with the CBM program’s “non-technological” designation, these types of ideas are no longer thought to be patentable. Inter partes review is the most-used of the proceedings created by the AIA and the one stakeholders we interviewed were most familiar with when they discussed the Patent Trial and Appeal Board. The other proceedings have been used less frequently, likely because of the short window for filing a challenge, in the case of post-grant review, and because of additional restrictions on what patents may be challenged, in the case of CBM. Under statute and regulation, the full review process at the Patent Trial and Appeal Board for any of the three proceedings generally takes up to 18 months and comprises two phases: (1) the petition phase, which lasts up to 6 months, and (2) the trial phase, which generally lasts up to 12 months. During the petition phase, the petitioner—typically a party accused of patent infringement, in the CBM program— files a petition challenging the validity of one or more of the patent’s claims and pays fees for each challenged claim. In some cases, a petitioner will file more than one petition challenging a patent. This might occur when a petitioner is constrained by the maximum number of pages allowed in a petition. Multiple petitions can also be filed against a single patent if the patent owner has sued more than one party for infringement, and each files a separate petition challenging the patent’s validity. Petitioners might also file a petition under more than one proceeding, either concurrently or sequentially. When a petition is received and the fees paid, administrative personnel of the board, under direction of the Chief Judge, assign three technically trained administrative patent judges to the case. According to agency documents, these three-judge panels are put together taking into account many factors, including technical experience, experience at the board, potential conflicts of interest, and availability. The patent owner may then, within 3 months of the petition date, file a preliminary response to the petitioner’s arguments. Within 3 months of submission of any preliminary response, or the last date on which such response may be filed, the panel of judges determines whether to allow the petition to move to the trial phase for review. This determination is called the “institution decision.” According to statute and regulations, in the case of the CBM program and post-grant review, a panel of judges may not institute a review unless the information presented in the petition, if not rebutted, would demonstrate that it is “more likely than not” that at least one of the claims challenged in the petition is unpatentable, or in the case of inter partes review, if the petitioner has a “reasonable likelihood” of prevailing. The first step in the trial phase is discovery (a step that exists in all federal civil litigation), during which the parties produce documents or testimony relevant to the challenged claims. Each party has 3 months to file discovery documents for the panel of judges’ review. If a petitioner and patent owner do not settle a case or it does not otherwise terminate, the case will proceed to the oral hearing. The hearing is an opportunity for the parties to make their strongest arguments and to answer judges’ questions, according to a board official, and after the hearing, the panel of judges will deliberate over the course of a few weeks or months and then issue its final written decision. The final written decision must be issued within 1 year of the institution decision, with limited exceptions. The patent owner may, for example, cancel one or more claims in the patent in an attempt to avoid institution of the trial. Figure 2, shows the progression of a case from the petitioner’s filing to the panel of judges issuing a final written decision. Under its Standard Operating Procedures, every Patent Trial and Appeal Board decision is, by default, a routine opinion until it is designated as “representative,” “informative,” or “precedential.” Representative decisions typically provide a representative sample of outcomes on a particular matter; they are not binding authority. Informative decisions provide norms on recurring issues, guidance on issues of first impression, and guidance on the board’s rules and practices; they are not binding authority. Precedential decisions are binding authority and emphasize decisions that resolve conflicts or address novel questions. Nominations for these designations can be made by a Patent Trial and Appeal Board judge, the Chief Judge, the Director of USPTO, the Deputy Director of USPTO, the Commissioner for Patents, or the Commissioner for Trademarks. Also, a member of the public may nominate a decision for a precedential designation within 60 days of its issuance. The Chief Judge can designate a nominated decision as representative or informative, but under Standard Operating Procedures, a precedential designation requires a majority agreement among all voting members of the board, including administrative patent judges and statutory members, as well as concurrence by the Director of the USPTO. Court Decisions on Eligibility for Review under the CBM Program Petitioners and patent owners may appeal the final written decisions of the Patent Trial and Appeal Board to the U.S. Court of Appeals for the Federal Circuit, just as unsatisfied plaintiffs or defendants may appeal a federal district court decision, and decisions may ultimately be appealed to the U.S. Supreme Court. The following decisions have significantly influenced the eligibility rules for CBM review, for different reasons: In Cuozzo Speed Technologies, LLC v. Lee (June 2016), the U.S. Supreme Court affirmed the board’s use of the “broadest reasonable construction” standard—meaning the ordinary meaning that someone skilled in the art would reach—to define the language of the claims during post-grant review as a reasonable exercise of the board’s rulemaking authority. Defining claim language using the broadest reasonable interpretation meant that the number of business method patents that could be determined as financial in nature is larger than it would otherwise be, so more patents are potentially eligible for review under the CBM program. In Unwired Planet, LLC v. Google Inc. (November 2016), the U.S. Court of Appeals for the Federal Circuit ruled that the USPTO’s policy of assessing whether a claim’s activities were “incidental” or “complementary” to a financial activity was too broad a standard to apply when determining whether a patent claim was eligible for a CBM review. The court stated that, to be CBM-eligible, a patent must claim a method used in the practice, administration, or management of a financial product or service. Applying this narrower standard effectively reduced the number of patents accepted for review under the CBM program. In Secure Axcess, LLC v. PNC Bank Nat’l Assoc. (February 2017), the U.S. Court of Appeals for the Federal Circuit clarified that a CBM patent must specifically have a claim that contains an element of financial activity in order for a patent to qualify for review under the CBM program. Like the Unwired Planet decision, the narrower standard expressed by the court has led to fewer patents being eligible for review under the CBM program. More Than 350 Patents Have Been Challenged under the CBM Program, and About One-Third of These Patents Were Ruled Unpatentable From September 2012 through September 2017, parties accused of patent infringement filed 524 petitions challenging the validity of 359 distinct patents under the CBM program, resulting in rulings against about one-third of these patents. The average monthly number of CBM petitions fluctuated during this period, but use of the program has declined since about 2015. Some stakeholders have expressed concern about multiple petitions being filed against the same patent, but our analysis of petition data showed that the vast majority of patents challenged under the CBM program were challenged once or twice. Overall, through September 2017, the Patent Trial and Appeal Board completed reviews of 329 of the 359 patents challenged under the program, and the board ruled at least some challenged patent claims unpatentable in about one-third of these patents. Petitioners Have Challenged the Validity of 359 Patents under the CBM Program, but Use of the Program Has Declined Overall Parties accused of patent infringement filed 524 petitions for patent review under the CBM program from September 2012 through September 2017, with the number of petitions per month fluctuating but tapering off over time (see fig. 3). During this 5-year period, an average of more than 9 petitions per month were filed under the CBM program, but this average rate has declined since 2015 to fewer than 5 per month in the last fiscal year, with no petitions filed in August or September 2017. As a point of comparison, the number of petitions for inter partes review has generally increased over the 5-year period. Stakeholders we interviewed suggested several possible reasons for the decline in CBM petitions. Specifically, some stakeholders told us that recent Federal Circuit and Supreme Court decisions that have changed what is patentable subject matter and the eligibility criteria for CBM review may have reduced the set of business method patents eligible for CBM review. Some stakeholders also suggested CBM petitioners successfully targeted the lowest-quality business method patents in the early years of the program, and now that those patents have been challenged, there are fewer patents that do not meet patentability requirements. Another possibility, according to stakeholders, is that owners of business method patents are wary of asserting their intellectual property and risking its invalidation, especially in light of the Alice decision, which effectively limited the patentability of some business methods. As a result, according to these stakeholders, fewer such patents end up in litigation and subsequently before the Patent Trial and Appeal Board. Some stakeholders also told us the CBM program has reduced patent infringement lawsuits, including some filed by non- practicing entities. In addition, a few stakeholders told us some patent owners may be waiting until after the CBM program sunsets to assert their patents. Patents Are Infrequently Challenged More Than Once or Twice Some stakeholders we interviewed were concerned about multiple petitions being filed against the same patents; however, our analysis showed that the vast majority of the 359 distinct patents challenged under the CBM program were challenged only once or twice under that program. Stakeholders have suggested that petitioners are, in some cases, using the CBM program and the inter partes review program as tools to increase costs borne by patent owners, and in the case of the CBM program, as a tool to delay district court proceedings. Some stakeholders have stated that the use of the AIA trials in this manner amounts to harassment, and at least one stakeholder has written letters to USPTO requesting the Director to intervene. However, our analysis of petition data showed that among the 359 patents challenged under the CBM program, 73.3 percent were challenged once and 18.4 percent were challenged twice during the 5- year period we reviewed. Another thirty patents, or 8.4 percent, were challenged more than twice under the CBM program during this period (see fig. 4). Of these 30 patents, in many cases multiple parties challenged a single patent; in others, a single petitioner or set of petitioners challenged a patent multiple times. In addition, of the 359 patents challenged under the CBM program during the 5-year period we reviewed, 92 were also challenged at least once in inter partes review. In some instances, petitioners filed concurrent petitions for CBM and inter partes review if, for example, they were unsure if the claims were eligible for a CBM review. In other instances, petitioners first sought CBM review and, when that was unsuccessful, filed an inter partes review. In these cases, petitioners may initially be seeking CBM review because of the additional grounds available for challenging the patents, and then turning to the inter partes review program if the CBM challenge proves unsuccessful. In other instances, petitioners first had success under the inter partes review program and then filed another petition under the CBM or inter partes review programs, according to our analysis of petition data. When including patent challenges under both the CBM and inter partes review programs, 52.1 percent of the 359 patents challenged under the CBM program were challenged once and 29.3 percent were challenged twice (see fig. 4). More than half of the patents challenged under both programs (50 of 92 patents) did not have any challenged patent claims instituted for trial under the CBM program, meaning that those patents, in many cases, did not meet the CBM program’s eligibility requirements and may have been more appropriately challenged with an inter partes review. There are several other reasons why petitioners may file more than one petition against a single patent, according to stakeholders we interviewed. First, the board limits the number of pages that a petitioner may use to submit prior art and arguments for invalidity. Some petitioners might file more than one petition so they have room to present all of their art and arguments at once. Data we analyzed on CBM petitions show that many follow-on petitions are filed on or near the same day as the first petition, supporting this argument. Second, in some cases the patent owner may not identify all the asserted patent claims in the district court right away or may change the set of asserted claims later in the proceedings, necessitating an additional CBM or inter partes review petition to cover the new claims. Third, in order to get the expensive district court proceedings stayed—that is, halted pending the board’s decision on the patent’s validity—a petitioner may file a CBM petition on patentability or clarity grounds soon after the district court trial commences, because these arguments require limited time to formulate. Later, once the petitioner takes the time to investigate the prior art, the petitioner might file a second petition challenging the patent for non-novelty or obviousness. In our analysis of petition data, we found some examples that were consistent with this approach. Fourth, if a patent owner charges multiple entities with patent infringement, each of the alleged infringers has an individual right to file a petition challenging the patent’s validity. The defendants in the infringement suits who become petitioners at the board may collaborate with one another and join their cases, but they may also choose to file petitions individually. In our analysis of petition data, we found examples of both. Petitioners might choose to join their cases in order to share the cost of counsel, while others may choose not to join their cases, perhaps because they use substantially different art and arguments in their petitions. Our analysis of the petition data found some examples of multiple petitions against a single patent that may raise questions about the legitimacy of the follow-on petitions. In some instances, a second, follow- on petition challenging the patent’s validity on the same statutory grounds as it did in the first petition was filed by the same petitioner after the first petition was denied institution. This type of multiple petitioning may occur when, for instance, a procedural termination resulted from a technical error in the first petition. Board officials said it may also occur because a petitioner is using the first denial of institution to alter the arguments and guide the second petition, a strategy that the board has labeled “road- mapping.” In other instances, a single petitioner filed a second, follow-on petition challenging the patent on different statutory grounds after the first petition was denied institution. These follow-on petitions may be legitimate attempts to correct simple errors in the first petitions, or they may reflect practices that might raise questions about whether the program is being used as intended. Patent Trial and Appeal Board officials are aware of concerns over multiple petitions and recently concluded a study about the prevalence of such practices in relation to all three types of proceedings created by the AIA. The board found that almost two-thirds (63.4 percent) of follow-on petitions were filed on or near the same day as the first petition. Nearly three in four (72.4 percent) follow-on petitions were filed before the institution decision on the first petition. These findings suggest that most petitioners are not waiting to use the board’s decision of non-institution as a guide for developing a second petition. Moreover, the board officials we interviewed told us they are empowered to deny a petition if they determine the petition presents the same or substantially the same prior art or arguments previously presented in another petition. Board officials told us they had denied several recent petitions on this basis. In addition, in a recent precedential opinion, the board clarified the characteristics it looks for to determine whether it should deny an inter partes review when a petitioner submits a follow-on petition. These characteristics include whether the petitioner previously filed a petition against the same patent claims; whether the petitioner provides adequate explanation for the time elapsed between filing two or more petitions against the same patent claims; and whether the petitioner knew, or should have known, about the prior art presented in the second petition at the time of the first petition. Claims Have Been Ruled Unpatentable in More Than One-Third of Patents Challenged under the CBM Program The Patent Trial and Appeal Board has ruled unpatentable some or all of the patent claims instituted for trial in about one-third of challenged patents and about one-third of petitions under the CBM program. Data on petition outcomes, however, are open to different interpretations depending on how they are presented. For example, board judges ruled some or all of the patent claims considered at trial unpatentable in 96.7 percent of petitions (175 of 181) under the CBM program for which they issued a final written decision from September 2012 through September 2017. On the basis of this statistic, the board could seem to invalidate the majority of the patents it reviews, as noted by some stakeholders. However, this outcome is predictable given the criteria for institution of a CBM trial—a judge panel will institute a petition to the trial phase if it is “more likely than not” that at least one of the claims challenged in a petition is unpatentable—which tips outcomes for instituted petitions toward rulings of unpatentability. In addition, board judges did not issue final written decisions for all petitions that enter the trial phase because the parties often reach a settlement before the final written decision. When taking into account all of the CBM petitions that had an outcome as of Sept 30, 2017, board judges ruled some or all of the claims considered at trial unpatentable in 35.6 percent of the cases (175 of 492). The results are similar when considered by patent rather than by petition. Specifically, for patents challenged between September 2012 and September 2017 and for which a final written decision was issued in at least one petition, 95.2 percent of patents (120 of 126) had some or all the patent claims that were instituted for trial ruled unpatentable. However, because not all challenged patent claims are instituted for trial and because final written decisions are not issued for all petitions that enter the trial phase, it is also accurate to say the board judges ruled some or all of the patent claims unpatentable for 36.5 percent of challenged patents (120 of the 329) that had an outcome as of September 30, 2017 (see fig. 5). Changes in petition outcomes over time also challenge the idea that the board invalidates most patents it reviews. In particular, the percentage of CBM petitions instituted for trial has decreased over time (see fig. 6). In 2012, about 80.0 percent of CBM petitions had some or all challenged claims instituted. In comparison, in 2016 about 53.5 percent of CBM petitions had some or all claims instituted. Preliminary data for 2017 suggests that this trend might continue: through September 2017, about 38.5 percent of CBM petitions had some or all claims instituted. Similar to the decline in number of petitions filed, this trend might have a few explanations, according to stakeholders. Specifically, board panels might be less likely to institute a petition for trial based on conclusions of the U.S. Court of Appeals for the Federal Circuit in Unwired Planet and Secure Axcess. Another possibility is that the patents in earlier cases represented the easiest targets for validity challenges, and thus the more recent challenges are based on shakier legal grounds and less likely to meet the CBM program’s institution threshold. In addition to declining institution rates, there has been an increase in the percentage of CBM petitions that settle before reaching an outcome. Specifically, the percentage of cases where the parties settled their dispute either before or after the institution decision increased from about 6.7 percent in 2012 to about 28.9 percent in 2016. When a case before the board is settled, it generally concludes any concurrent district court infringement case. The patent owner’s intellectual property remains in place, and the patent owner is free to assert the patent against other alleged infringers later. The Board Met Timeliness Requirements and Has Taken Steps to Analyze Decisions and Improve Proceedings but Does Not Have Guidance to Ensure Decision Consistency The Patent Trial and Appeal Board has completed all trials under AIA- authorized proceedings within statutorily directed time frames, according to board data, and the board has taken steps to review issues that could affect the consistency of its trial proceedings and decisions and to engage with stakeholders to improve its proceedings. To ensure timeliness of trial proceedings, the board provided a checklist of information and time frames to petitioners and patent owners, among other things. According to board documents and interviews with officials, the board has also taken steps to review and assess its trial proceedings and decisions, but it does not have guidance for reviewing trial decisions, or the processes that lead to the decisions, for consistency. The board has also taken several steps to engage with stakeholders regarding various aspects of trial proceedings. Patent Trial and Appeal Board Data Indicate Trials Have Been Completed within Statutorily Directed Time Frames According to data on Patent Trial and Appeal Board proceedings, as of September 31, 2017, all trials under AIA-authorized proceedings, including the CBM program, have been completed within statutorily directed time frames. The board maintains a database of trial proceedings that includes the date of each petition, decision to institute a trial, and final written decision. Board officials we interviewed told us the timeliness of decisions to institute a trial and of final written decisions has not been a concern in the 5 years that it has operated. According to board officials, as of November 2017, two AIA trials—one under the inter partes review program and one under the CBM program—have been extended for good cause past the typical 1-year time limit between the institution decision and the final written decision, as allowed by statute. Board officials told us they have taken several steps to ensure that trials are completed within required time frames. According to board documentation, between 2012 and 2017, for example, the board hired more than 150 additional administrative patent judges, in part to preside over AIA trials. In addition, the board has taken several proactive administrative steps to help ensure that stakeholders are aware of requirements for information filing and dates. For example, when a petition is filed, the board’s administrative staff creates a checklist of information required and due dates, and communicates these dates and requirements to petitioners and patent owners throughout the trial. Some stakeholders have expressed concern that AIA trial time frames are too short and deprive patent owners and petitioners of due process rights. One patent attorney that we spoke with, for example, noted that the short time frames limit discovery. As directed by the AIA, a final determination for a review generally must be issued not later than 1 year after the date a review has been instituted, and the director may extend that period by up to 6 months for good cause. Board officials we interviewed stated that they do not believe parties are having trouble completing discovery activities in the time allotted in view of the limited discovery allowed at the board. Board officials further stated that they have not found compelling reasons to extend trial proceedings on the basis of the need for additional discovery. As reflected in USPTO’s strategic plan, timeliness of the board’s trial process is a key program goal, and board officials said trials would be extended only in unusual circumstances. In addition, board officials stated that the board adheres to the 12-month timeline for final written decisions because this timeline gives the district courts a definitive and predictable endpoint for the trials. The Board Has Taken Several Steps to Review Issues That Affect Trial Proceedings, but It Does Not Have Guidance to Ensure the Consistency of Its Decisions The Patent Trial and Appeal Board has decision review processes that help ensure trial decisions are revisited as appropriate, but the board cannot ensure the consistency of these decisions because it does not have guidance for reviewing them or the processes that lead to them. For trials still in progress, board officials told us that there are several ways that management gets involved in reviews. According to officials, a review of an ongoing trial is triggered if and when a paneled judge raises any issue deserving of management attention. Such issues are brought to the attention of the Chief Judge or other members of the board’s management team and are acted upon at their discretion. According to board officials, the usual response is a management meeting with the three-judge panel, with the goal of ensuring the judges are aware of any precedent or ongoing trials dealing with similar issues. The officials said these review meetings are also meant to ensure that board management is aware of any decisions that may be relevant to the stakeholder community or the public. According to board officials, issues that may prompt action include those that are not routine in nature, that involve novel questions of law, or that may result in decisions that could contradict previous board decisions. Board officials called these review meetings the first step for keeping track of key issues. Board officials told us these reviews raise a fair number of issues, but the process relies on self-reporting by the judges, and board officials told us the effectiveness of these reviews is not measured. Board officials also told us that a separate internal review process has evolved over time, whereby a small group of board judges, in consultation with board management, seeks to ensure decision quality and consistency by reading a large number of draft AIA trial decisions and giving feedback or suggestions to authoring judges prior to issuance. The board is currently drafting a formal charter that will outline the group’s function, reviewer selection, and membership term. According to board officials, these reviews are meant to help ensure consistency with applicable board rules, other board decisions, and Federal Circuit and Supreme Court case law. In addition, such reviews may result in coaching and training to increase an individual judge’s quality of performance. Regarding completed trials, board officials told us they review any board AIA trial decisions that are appealed to the U.S. Court of Appeals for the Federal Circuit and that the appeals court reverses or remands. Specifically, the board monitors Federal Circuit decisions and board management then reviews any reversals or remands for opportunities to improve processes and stay abreast of emerging issues. According to board officials, for any reversal or remand, board management and members of the three-judge panel that decided the case meet to discuss what steps could have been taken to avoid the Federal Circuit reversal or remand, and what else can be learned from the Federal Circuit decision. In some instances, according to officials, the board will host a session where all board judges are invited to review and discuss the trial court decision and the decision of the Federal Circuit. In addition, board officials told us they track data on Federal Circuit affirmances, remands, and reversals. The board has recently updated its Standard Operating Procedure to provide guidance on how it handles cases remanded by the Federal Circuit. This procedure creates internal norms to promote timeliness and consistency of the board’s response to remands. The procedure includes a goal for the board to issue decisions on remands within 6 months of receipt and calls on the Chief Judge and the Deputy Chief Judge to discuss each remanded case with the presiding three- judge panel before the panel expends substantial effort on the case. The Chief Judge may also elect to expand the panel assigned to the remanded case, when deemed prudent. Furthermore, officials told us that all board decisions—including final written decisions, decisions to institute a trial, and any substantive orders—are reviewed by board judges on the date of issuance. Specifically, a rotating group of judges, on a voluntary basis, reads and analyzes each day’s decisions and, according to board officials, sends a summary list of the number of decisions made that day along with a brief decision summary for any cases where key issues of interest were raised. Board officials said that most decisions are straightforward and generally not summarized in detail. For decisions highlighted in the summary report, according to officials, a lead judge, in most cases, will then review the decision more closely. Example summary lists provided to us by the board show brief summaries of a trial involving interpretations of prior art admissibility and a trial dealing with an interpretation of a challenge based on clarity. Finally, board officials told us that the board has begun to increase the number of trial decisions considered for precedential and informative designations as part of its efforts to ensure the consistency of trial decisions. Board officials also told us that increasing the number of these designations had not been a priority while the AIA trial procedures and processes were being operationalized and as the board was hiring more than 150 administrative patent judges over the past 5 years. However, officials said that they are now taking steps to simplify the vetting and voting process, and the board expects more precedential and informative designations going forward. Taken together, the board’s review processes help ensure that board trial decisions are reviewed in some manner. However, because the board does not have documented procedures for how to review decisions for consistency, the board cannot fully ensure the consistency of the decisions or the processes that lead to them. USPTO’s 2014-2018 strategic plan includes the goal to “optimize patent quality and timeliness,” which includes an objective to “maintain ability to provide high-quality decisions.” As part of this objective, the plan states that it is “critical for the to ensure consistency in its decisions through review of decisions in proceedings.” Under federal standards for internal control, management should design control activities to achieve objectives and respond to risks. Such control activities include clearly documenting internal control in a manner that allows the documentation to be readily available for examination. The documentation may appear in management directives, administrative policies, or operating manuals. However, the board has not yet clearly documented how judges are to review trial decisions, or the processes that lead to the decisions, to ensure consistency. Without developing guidance, such as documented procedures, outlining the steps USPTO will take to review the Patent Trial and Appeal Board decisions and the processes that lead to decisions, USPTO cannot ensure that it is fully meeting the objective of ensuring consistency of its decisions. The Patent Trial and Appeal Board Has Taken Several Steps to Engage Stakeholders and Address Stakeholder Concerns The Patent Trial and Appeal Board has taken several steps to engage stakeholders regarding trial proceedings and decisions and address related concerns. USPTO’s strategic plan states that the board should expand outreach to stakeholders by providing opportunities for interaction and updates on board operations and other important issues. The board has done so through several types of public outreach efforts, including participating in roundtables, webinars, and judicial conferences, among other activities. The board has made several changes to policies and procedures based on stakeholder feedback gathered through these mechanisms. For example, after the Patent Trial and Appeal Board had been operational for about 18 months, it conducted a series of eight roundtables in April and May of 2014 at locations around the country to publicly share information concerning trial proceedings, to obtain public feedback on these proceedings, and to launch the process of revisiting its trial rules and trial practice guide. At these roundtables, the board provided the public with statistics summarizing the administrative trial proceedings, as well as lessons learned for filing effective petitions, engaging in successful discovery and amendment practice, and effectively presenting a case at oral hearing, among other things. The board also asked for and received feedback from the public on the AIA administrative trial proceeding rules and trial practice guide, as well as on experiences in general with the AIA administrative trial proceedings. Subsequent to the 2014 roundtables, the USPTO sought public input on all aspects of AIA trial proceedings through a June 27, 2014 Federal Register notice, which included 17 specific questions regarding certain trial rules, such as claim construction, the claim amendment process, and good cause trial extensions. USPTO took a two-step approach in responding to the 37 comments received in response to this Federal Register notice. First, USPTO implemented several immediate changes to board proceedings, including changes to page limits for some documents. According to the annual report of USPTO’s Patent Public Advisory Committee, these changes were favorably received by the stakeholder community. Second, in April 2016, the board implemented more substantive changes, including allowing testimonial evidence to be submitted with a patent owner’s preliminary response to a petition and changing from a page limit to a word count for major briefings, among other things. In addition to roundtables, the board has engaged with stakeholders through several other mechanisms, including webinars and judicial conferences. For example, in February 2015, the board announced its inaugural “Boardside Chat” lunchtime webinar series, which has been held bi-monthly ever since. These webinars are designed to update the public on current board activities and statistics, and to allow a means for the board to regularly receive public feedback about AIA trial proceedings and any issues of concern. Topics discussed at these events include key trial decisions, proposed changes to trial rules, and best practices for prior art presentations in AIA trials, among other things. Since 2015, the board has hosted an annual judicial conference, where the board engages with stakeholders and educates them about AIA trial proceedings, answers questions, and receives feedback. Board judges present trial statistics, information about the internal functioning of the board, practice tips, and engage in discussions on topics of current interest to stakeholders. Topics have included motions to amend and the prevalence of multiple petitions. More recently, the board has conducted other outreach sessions, including: an August 2017 roundtable meeting with stakeholders from the American Intellectual Property Law Association to address a broad range of topics affecting practitioners before the board, including how patent claims are interpreted, claim amendments, and conditions under which multiple petitions from a single petitioner would be denied; a webinar on August 31, 2017, addressing common evidentiary issues that occur during AIA trial proceedings; and a webinar on September 12, 2017, with the Chief Judge to commemorate the 5th anniversary of the board, where discussion topics included the origins and mission of the board, recent board developments, and operational procedures. According to USPTO’s Patent Public Advisory Committee, this type of outreach provides a valuable two-way conduit for constructive flow of information to and from the board. In addition to these various outreach efforts, stakeholders are encouraged to provide feedback to the board, on any topic related to trial proceedings, by e-mail or telephone. Board officials we interviewed told us that they review information obtained from stakeholders during roundtable meetings and other outreach events and implement changes to policies and procedures where applicable. The officials told us that stakeholder feedback has been used to inform updates to the board’s trial rules guidance, to modify rules of practice, and in updating Standard Operating Procedures. In addition, board officials told us that in response to stakeholder concerns, they conducted two extensive studies covering motions to amend and the filing of multiple petitions against a single patent. Furthermore, board officials told us that they have held training sessions for judges regarding specific areas of interest to stakeholders. Lastly, board officials also told us that the board’s website, including the frequently-asked-questions pages, is updated with information relevant to stakeholders, including stakeholder concerns. For example, written stakeholder comments submitted in response to a proposed rulemaking are posted on the USPTO website for public viewing. Stakeholders Agree the CBM Program Has Reduced Litigation, and Many See Value in Maintaining Aspects of the Program Stakeholders we interviewed generally agreed that the CBM program has reduced litigation, and many said there is value in maintaining some aspects of the program. Stakeholders generally agreed that the CBM program has contributed to a decrease in litigation involving business methods patents and that the program has had positive effects on innovation and investment. Most stakeholders also said there is value in maintaining, among other things, the ability to challenge patents on all four statutory grounds before the Patent Trial and Appeal Board. Stakeholders Generally Agreed the CBM Program Has Contributed to a Decrease in Litigation Involving Business Method Patents Stakeholders we interviewed generally agreed the CBM program has reduced litigation involving business method patents because the CBM program allows these patents to be more easily challenged than in district courts. Stakeholders told us that fewer business method patent lawsuits are filed and that existing lawsuits are often dropped after patents have been through the CBM program. However, stakeholders also noted that the Supreme Court’s 2014 decision in Alice may have also reduced the number of business method patent lawsuits. Patents that would be found invalid under Alice are often very similar to the patents that are eligible for challenge under the CBM program, and in some cases, according to stakeholders, it is cheaper and more efficient to challenge a patent’s validity in district court using Alice than it is to use the CBM program. Stakeholders described the following additional effects of the CBM program: Business method patent assertion is riskier. The CBM program makes it riskier to assert business method patents because, compared with district court, the program offers a cheaper and more efficient way for alleged infringers to challenge a patent’s validity. District court litigation can take several years and cost several million dollars, while CBM trials are limited to 18 months and generally cost much less. In addition, technically trained board judges have greater expertise in patent law than an average district court judge and jury, and are often better able to understand complex patentability issues. Because of this, some alleged infringers are more willing to present complex arguments—such as questions about whether the patent meets standards for clarity—to the board than to a jury. As a result, the CBM program has deterred owners of financial business method patents from asserting their patents for fear those patents will be ruled unpatentable. According to stakeholders, the existence of CBM challenges has put downward pressure on settlement amounts. Patent owners may want to avoid the risk of their patent being invalidated and will demand lower settlement amounts to avoid the risk of CBM and district court proceedings. Petitioners, too, told us they use this knowledge to negotiate lower settlement fees. In addition, because challenges under the CBM program may suspend the parallel district court proceedings, it is more difficult for patent owners to expect quick settlements from alleged infringers looking to avoid the rapidly increasing court costs associated with lengthy trials. The parties can still reach settlements after the alleged infringer files a challenge under the CBM program, but the patent owners have less leverage in negotiations. On the other hand, for patent owners willing to go through a CBM challenge, their patents will emerge stronger having survived the additional review according to stakeholders we interviewed. Business method patent owners have adjusted assertion strategies to avoid the CBM program. Patent owners are focused on asserting business method patents that are higher quality and less vulnerable to challenge under the CBM program or based on the Supreme Court’s decision in Alice; in other words, those patents that describe a technological invention that is not abstract and implemented on a generic computer. In addition, a few stakeholders told us that they have abandoned some claims in certain patents to avoid the possibility of their patents being challenged under the CBM program. Stakeholders also told us that patent owners seem to be asserting more patents, and more claims, than before the CBM program was implemented, as a strategy either to ratchet up defense costs for accused infringers and secure a settlement or to at least have success with some of the infringement charges. In addition, some stakeholders said that because the board charges fees for each petition challenging a patent, asserting more patents is a strategy to increase expected costs of defending against infringement and, thus, to increase the likelihood of a settlement. However, our analysis of RPX litigation data from 2007 to 2017 did not support these assertions. Patent litigation data did not show an increase in the monthly average number of patents asserted per case among cases involving one or more business method patents. The CBM program has decreased the value of business method patents. The CBM program has decreased the value of business method patents generally, even beyond those focused on financial services. Several stakeholders told us that the board’s broad initial interpretation of the CBM program’s eligibility requirements contributed to an increased risk to a wider swath of business method and software patents than was intended by Congress. Stakeholders told us that any patent tangentially related to financial business methods has been devalued because it could potentially be challenged under the CBM program. In addition, stakeholders said they believed that the threat of such challenges has decreased the value of all business method patents, including those that might ultimately survive a CBM challenge. Some stakeholders pointed to a decrease in licensing of business method patents and others suggested that patents have lost value on the secondary patent market. Available data that we reviewed, though limited, support the claims that patent values on the secondary market have fallen. A few stakeholders, however, told us that to the extent these patents have lost value, the devaluation is related to problems with patent quality. Stakeholders Generally Agreed the CBM Program Has Had Positive Effects on Innovation and Investment Stakeholders generally agreed the effects of the CBM program on innovation and investment have been minimal or mostly positive. More specifically, stakeholders told us that the CBM program is good for overall innovation and investment in financial technologies in that the program eliminates overly broad (non-specific), low-quality patents. Stakeholders told us they believe the existence and assertion of overly broad patents is bad for innovation, in part because defending against alleged infringement is expensive and time-consuming, even under the CBM program. Assertion of overly broad, unclear, or otherwise low-quality patents acts much like a tax on investment, according to stakeholders. Stakeholders also told us that removing such patents from the marketplace promotes innovation because it prevents these patents from blocking new innovation. According to stakeholders, innovation is represented by the quality of the patents issued rather than the quantity. A large number of patents in a technology space, according to stakeholders, can make it difficult to innovate within that crowded space. A few stakeholders had differing views, stating that the CBM program has affected some companies’ ability to protect a business model with a business method patent, although one stakeholder acknowledged that the Supreme Court’s decision in Alice has also had an effect. These types of comments were generally from stakeholders with company-specific interests, including individual patent owners and companies that have had patents invalidated under the CBM program. Other stakeholders, however, including those in the financial services industry, told us that innovation in their field is robust. For example, these companies are developing mobile-payment and blockchain technologies, and the companies have not seen any negative effects from the CBM program on their ability to innovate, patent, and invest in these financial services technologies. Stakeholders generally agreed that the CBM program and the other post- grant programs have had a positive effect on patent quality, as patent applicants are more and more aware of what it takes to ensure a patent will survive a post-grant challenge. Several stakeholders highlighted extra steps they have taken before and during the patent application and examination stages to ensure their patents will stand up to any eventual challenges. For example, one patent owner told us how his company proactively worked to get its patent examined by a foreign patent office, in an effort to understand any quality issues with the patent, before submitting a patent application to USPTO. Another stakeholder told us about an extended back-and-forth with the USPTO examiner. This stakeholder told us that the additional effort taken during the examination process resulted in a patent that is much clearer and that will be more likely to stand up to additional scrutiny. Most Stakeholders Said There Is Value in Maintaining Aspects of the CBM Program Most stakeholders told us there was value in maintaining aspects of the CBM program, including the ability to challenge patents on all four statutory grounds at the Patent Trial and Appeal Board, and many told us that it would be useful to expand this capability to a broader set of patents beyond business methods. However, there was no strong consensus among stakeholders for how the AIA trials should be designed in the future. Stakeholders generally agreed that the ability to challenge a patent’s validity on subject matter eligibility grounds remains important, although there was not broad agreement among stakeholders regarding how far that ability should extend beyond business method patents. Stakeholders we interviewed pointed to inconsistencies in how federal courts interpret subject matter eligibility requirements and said that challenges on subject matter eligibility grounds should remain an option at the Patent Trial and Appeal Board because of the board’s expertise over the courts. Some stakeholders said subject matter eligibility challenges were important for a wider scope of patents than just business methods because concerns about subject matter eligibility that apply to business method patents extend to software-related patents in general. In addition, a few stakeholders suggested that subject matter eligibility challenges should be available for patents in all areas of technology. The continued prevalence of challenges in district courts based on the Supreme Court’s decision in Alice, for business method patents and for a wider array of patents, highlights the importance of retaining the ability to challenge patent validity at the board on subject matter eligibility grounds. Similarly, stakeholders told us that patent clarity problems exist beyond business method patents. Stakeholders said that the federal courts and jurors do not necessarily have the expertise to interpret patent clarity requirements and that the technically trained Patent Trial and Appeal Board judges were better suited to make patentability determinations, including on clarity grounds. One stakeholder, for example, told us that petitioners can delve much deeper into the invalidity argument on patent clarity grounds at a CBM trials than they can as defendants in district court, mostly because the board judges have the requisite technical expertise. In addition, many stakeholders told us that challenging patents on clarity grounds was also important for a much broader array of patents than business method patents, and some suggested that these challenges should remain an option for all patents challenged at the board. In June 2016, we reported that more than 40 percent of patent examiners experience pressure to avoid rejecting a patent application because of problems with clarity and we recommended additional steps USPTO could take to improve patent clarity. This suggests there are a potentially large number of patents, beyond and including business method patents, that could benefit from a second look by the board on these grounds, and inter partes review does not allow patents to be challenged on clarity grounds. Stakeholders discussed several other topics related to the future of the CBM program: Post-grant review is not an effective substitute for the CBM program for challenging patents on subject matter eligibility and patent clarity grounds. Stakeholders told us that the 9-month window, after a patent is issued, to file challenges using post-grant review is too short to make it an effective substitute for the CBM program. Post-grant review was established as a permanent mechanism at the board for challenging all patents on all statutory grounds. However, only 78 petitions have been filed for post-grant review through September 30, 2017. According to stakeholders, few companies have the resources to continuously monitor patent issuance in real time. In addition, even if companies do discover patents that are relevant to their business, companies, in general, are not willing or able to spend resources challenging patents that may never be used as the basis for an infringement lawsuit. As a result, the public essentially does not have the ability to challenge most patents on subject matter eligibility and clarity grounds, according to stakeholders. CBM challenges should not be limited to a specific technology. Although the CBM program was designed to address a problem caused by a narrow set of patents, some stakeholders told us they are troubled by CBM’s focus on patents for financial services and products. Stakeholders said that singling out such services and products is unfair and that the need to determine eligibility for review created uncertainty for patent owners. In addition, some stakeholders told us that the singling out of a particular subset of patents may raise questions about compliance with an international treaty. Concerns remain about business method and software-related patents. Some stakeholders told us the patents that the CBM program was designed to address have largely been addressed by improved examination at USPTO, reducing the need for the program. In addition, some stakeholders told us that the CBM program, which was designed to be temporary, had largely succeeded in addressing the problems with business method patents. However, other stakeholders told us that patents of questionable validity, including business method and software patents, continue to be issued by the patent office. Given these continuing concerns over software-related patents, several stakeholders suggested that one viable option for the future of the CBM program is to expand its eligibility beyond financial services patents to cover all software-related patents. In addition, in contrast to the inter partes review program, the CBM program allows any form of prior art to be used to challenge a patent on novelty or obviousness grounds. This broader allowance for prior art is important because many software and business method patents were preceded by prior art not found in existing patents or printed publications. Conclusions In 2016, we reported on a number of patent quality challenges at USPTO and made several recommendations to help improve the quality and clarity of issued patents. In that report, we estimated that almost 70 percent of patent examiners did not have enough time to complete a thorough examination of patent applications given a typical examiner’s workload. Given these time constraints and other patent quality challenges, the Patent Trial and Appeal Board has provided a means to challenge low-quality patents after they have been issued. Stakeholders generally agreed that the CBM program has reduced lawsuits in the federal courts involving business method patents, and many stakeholders were in favor of maintaining aspects of the program. The board has a track record of issuing timely decisions that have largely been upheld by the U.S. Court of Appeals for the Federal Circuit. However, the board does not have guidance, such as documented procedures, for reviewing trial decisions and the processes that led to the decisions. Without developing guidance, such as documented procedures, that outlines the steps USPTO will take to review the Patent Trial and Appeal Board’s decisions and the processes that lead to decisions, USPTO cannot fully ensure that it is meeting the objective of ensuring consistency of its decisions. Recommendation for Executive Action We are making the following recommendation to USPTO: The Director of USPTO should develop guidance, such as documented procedures, for judges reviewing the Patent Trial and Appeal Board’s decisions and the processes that lead to the decisions. (Recommendation 1) Agency Comments We provided a draft of this report to the Department of Commerce for review and comment. In its comments, reproduced in appendix II, the department agreed with the recommendation and stated that it has begun taking steps to address it, including drafting a formal, written charter that documents procedures for reviewing board decisions. The department further stated that it intends to address the recommendation within one year. In addition, it provided technical comments, which we incorporated as appropriate. As agreed with your office, unless you publicly announce the contents of this report earlier, we plan no further distribution until 8 days from the report date. At that time, we will send copies of this report to the appropriate congressional committees, the Secretary of Commerce, and other interested parties. In addition, this report will be available at no charge on the GAO website at http://www.gao.gov. If you or your staff members have any questions about this report, please contact me at (202) 512-3841 or neumannj@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix III. Appendix I: Objectives, Scope, and Methodology Our objectives were to (1) describe the extent to which the Patent Trial and Appeal Board’s Transitional Program For Covered Business Method Patents (CBM program) has been used to challenge patents, and the results of those challenges; (2) examine the extent to which USPTO ensures timeliness of trial decisions, reviews decisions for consistency, and engages with stakeholders to improve its administrative proceedings for the program; and (3) discuss stakeholder views on the effects of the CBM program and whether it should be extended past its scheduled September 2020 sunset date. To describe the extent to which the CBM program has been used to challenge patents, and the results of those challenges, we obtained data on board proceedings from two companies—RPX Corporation and Unified Patents—that included information on all of the board’s proceedings from September 2012 through September 2017. RPX and Unified Patents collect, compile, and analyze data from the U.S. Patent and Trademark Office’s publicly available data system. Both companies manually review these data to verify variables and to manually code additional information from other publicly available board documents. We conducted data quality testing, interviewed relevant officials, and reviewed relevant documentation for the data. We found these data to be sufficiently reliable for the purposes of our reporting objectives. For petitions filed at the board, data from RPX and Unified Patents include information on the patent in dispute, including its U.S. patent number, petition-filing dates, and trial institution and final written decision dates. RPX data include the patent claims challenged and the statutory grounds on which they were challenged. In addition, RPX data includes which patent claims were instituted for trial on which statutory grounds, and which patent claims were ruled unpatentable on which statutory grounds. RPX and Unified Patents provided the names of the petitioners and patent owners, as well as whether the patent owner is an operating company or one of several classifications of non-practicing entities. RPX also provided the names of the parties’ attorneys. We categorized which program each petition was filed under (CBM, inter partes review, or post- grant review) to enable comparisons across programs. We used the data from Unified Patents on Patent Trial and Appeal Board proceedings to supplement the RPX data for outcomes of each petition. Specifically, we compared the Unified Patents’ outcome variable—which describes the final outcome of the proceeding—and the RPX outcome variable to create a new variable that reflects the full available information about each petition’s outcome. There were some—fewer than 3 percent of cases—where the two variable values were inconsistent with one another. In these cases, we reviewed trial documentation to determine the correct value for the outcome variable. The Unified Patents outcome variable sometimes had more information than the RPX variable. For example, cases that were terminated because of settlement were identified as settlements in the Unified Patents data, but not in the RPX data. We retained the additional detail for our analysis. To determine trial outcomes at the patent level, we analyzed the petition in which the patent proceeded the furthest in the CBM process. For example, if a patent was challenged under the CBM program multiple times—for example, three times—and two petitions were not instituted to the trial phase and one was instituted and then settled before the board judges issued a final written decision, we used the petition that proceeded the furthest for our patent-level analysis of outcomes. In this way, we were able to report what happened to patents under the CBM program, while not double-counting those patents that were challenged more than once. To examine the extent to which USPTO ensures trial timeliness, reviews past decisions for consistency, and engages with stakeholders to improve its administrative proceedings for the program, we reviewed the America Invents Act (AIA); USPTO’s strategic plan; the Patent Trial and Appeal Board’s policy and guidance documents, including the Trial Practice Guide; and we interviewed board officials on several occasions. We compared USPTO’s efforts to review decisions for consistency against USPTO’s current strategic plan as well as Standards for Internal Control in the Federal Government (commonly referred to as the “Green Book”). In addition, we reviewed publicly available information documenting the steps the board takes to engage with stakeholders, including documentation of webinars, judicial conferences, and roundtable discussions. To obtain stakeholder views on the effects of the CBM program and whether it should be extended, we conducted semi-structured interviews with 38 stakeholders knowledgeable about the CBM program. To identify these stakeholders, we first identified the following sets of stakeholder groups: petitioners and patent owners who have been involved with CBM trials; attorneys who have represented clients with board proceedings; industry trade groups; academic and legal commentators; public interest groups; and venture capitalists. We identified petitioners, patent owners, and attorneys who had been involved in board proceedings using data from RPX Corporation and Unified Patents. We ranked petitioners, patent owners, and attorneys based on how many CBM cases they had been involved with, and how many inter partes review cases they had been involved with in front of the board. We then requested, via email, interviews with several stakeholders from each stakeholder group, and began our semi-structured interviews as stakeholders accepted our invitation. During our initial set of semi-structured interviews, we identified additional stakeholders through an iterative process known as a “snowball selection method,” whereby during each interview we solicited names of additional stakeholders it would be useful to interview. As we obtained the names of additional stakeholders, we requested additional interviews, conducted interviews, and solicited additional stakeholders, until we (a) had interviewed four or more stakeholders from each identified stakeholder group and (b) found that stakeholder responses were, in general, commonly describing the same broad themes and relevant points that previous stakeholders had described about the topics we were discussing. In total, the stakeholders we recruited and interviewed did not form a random, statistically representative sample of all relevant stakeholders. As such, we cannot generalize the results of the interviews. However, these stakeholder groups and the stakeholders we interviewed provide a broad spectrum of informed opinions on the CBM program. Of the 38 stakeholders interviewed, 14 had previously petitioned CBM against more than one patent owner, and many of those had also petitioned an inter partes review. In addition, we interviewed 6 patent owners that had been involved in multiple CBM trials. We also interviewed attorneys from 5 law firms that have represented multiple petitioners and patents owners in CBM cases. In addition, we interviewed officials from 4 trade groups, 4 venture capital firms, and 5 academics and legal commentators, all of whom had interest and expertise in the CBM program. During our semi-structured interviews, we asked stakeholders the following three broad questions: How much and in what way has the existence of the CBM program affected patent assertion strategies since 2012? How much has the CBM program influenced investment decisions and innovation for technologies related to financial-services business methods? Should the CBM program be allowed to expire in September 2020 or should it be renewed? For each question, we used a consistent set of follow-up prompts to ensure that we fully covered all aspects of each topic with the stakeholders, that we received complete answers, and that we were able to accurately record the responses. While we asked every stakeholder each of the three questions, we did so keeping in mind the particular background and experience of each stakeholder because experience and expertise differed across our wide range of stakeholders. As such, during each interview, we focused on the topics where the stakeholder had the most experience, expertise, or knowledge. To systematically analyze the information we collected during our semi- structured interviews, we used qualitative analysis software to group the responses into categories and themes. All information was individually coded by two analysts. We classified individual responses according to these broad themes, which generally corresponded to our main questions: The effect of the CBM program on patent assertion and litigation. The effect of the CBM program on innovation and investment in business methods. The future of the CBM program. Within each broad theme, we labeled and organized sub-themes. We established the sub-themes by identifying natural clusters of stakeholder responses. We analyzed the categorized themes and sub-themes to draw inferences about the effectiveness of the CBM program by taking the following steps: We first examined the amount and nature of agreement and disagreement between responses within each theme and sub-theme. We then assessed the strength of the arguments supporting each categorized response, and considered factors including the number of stakeholders who discussed a topic, including the strength of the rationale for each viewpoint and other supporting evidence provided. We also considered the way in which stakeholders’ interests could influence their perspectives. In this report, we present the themes with the strongest and most consistent support based on rationale including the prevalence of each argument, the presence of credible evidence in support of statements, and the amount of consistency and corroboration of themes across stakeholders. Because stakeholders do not make up a defined population that we could sample from, and because the stakeholders we interviewed had a wide range of experience and expertise, we did not tally up similar responses and do not present stakeholder responses based solely on how many stakeholders agreed or disagreed with a given statement. We conducted this performance audit from November 2016 to March 2018 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient and appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Appendix II: Comments from the Department of Commerce Appendix III: GAO Contact and Staff Acknowledgments GAO Contact Staff Acknowledgments In addition to the contact named above, the following individuals made contributions to this report: Rob Marek (Assistant Director), Kevin Bray, Mark Braza, Richard Burkard, Stephanie Gaines, Michael Krafve, Cynthia Norris, Ardith Spence, Sara Sullivan, and Sarah Williamson. Related GAO Products Intellectual Property: Patent Office Should Define Quality, Reassess Incentives, and Improve Clarity. GAO-16-490. Washington, D.C.: June 30, 2016. Intellectual Property: Patent Office Should Strengthen Search Capabilities and Better Monitor Examiners’ Work. GAO-16-479. Washington, D.C.: June 30, 2016. Intellectual Property: Assessing Factors That Affect Patent Infringement Litigation Could Help Improve Patent Quality. GAO-13-465. Washington, D.C.: August 22, 2013. U.S. Patent and Trademark Office: Performance Management Processes. GAO-10-946R. Washington, D.C.: September 24, 2010. Intellectual Property: Enhanced Planning by U.S. Personnel Overseas Could Strengthen Efforts. GAO-09-863. Washington, D.C.: September 30, 2009. Check 21 Act: Most Consumers Have Accepted and Banks Are Progressing Toward Full Adoption of Check Truncation. GAO-09-8. Washington, D.C.: October 28, 2008. U.S. Patent and Trademark Office: Hiring Efforts Are Not Sufficient to Reduce the Patent Application Backlog. GAO-08-527T. Washington, D.C.: February 27, 2008. U.S. Patent And Trademark Office: Hiring Efforts Are Not Sufficient to Reduce the Patent Application Backlog. GAO-07-1102. Washington, D.C.: September 4, 2007. Intellectual Property: Improvements Needed to Better Manage Patent Office Automation and Address Workforce Challenges. GAO-05-1008T. Washington, D.C.: September 8, 2005. Intellectual Property: Key Processes for Managing Patent Automation Strategy Need Strengthening. GAO-05-336. Washington, D.C.: June 17, 2005. Intellectual Property: USPTO Has Made Progress in Hiring Examiners, but Challenges to Retention Remain. GAO-05-720. Washington, D.C.: June 17, 2005.
target: Patents can promote innovation by giving inventors exclusive rights to their inventions, and patent owners can bring infringement lawsuits against anyone who uses, makes, sells, offers to sell, or imports a patented invention without authorization. As GAO previously reported, such lawsuits can take years and cost several million dollars. USPTO's CBM program provides a trial proceeding to challenge a patent's validity at USPTO's board for, according to stakeholders, a fraction of the time and money that would be spent in the federal courts. The CBM program began in September 2012 and is slated to sunset in September 2020. GAO was asked to examine the CBM program. This report (1) describes the extent to which the program has been used to challenge patents, and the results of those challenges; (2) examines the extent to which USPTO ensures timeliness of trial decisions, reviews decisions for consistency, and engages with stakeholders to improve proceedings for the program; and (3) discusses stakeholder views on the effects of the program and whether it should be extended past its sunset date. GAO analyzed CBM trial data from September 2012 through September 2017, reviewed USPTO documents, and interviewed 38 stakeholders, such as legal and academic commentators, selected for their knowledge of or direct involvement in such trials. From September 2012 through September 2017, entities facing patent infringement lawsuits filed 524 petitions challenging the validity of 359 patents under the U.S. Patent and Trademark Office's (USPTO) covered business method (CBM) program, resulting in decisions against about one-third of these patents. The CBM program provides entities facing infringement lawsuits an opportunity to challenge the validity of a business method patent by demonstrating that it did not meet requirements for patentability. Business method patents focus on ways of doing business in areas such as banking or e-commerce. The rate of filing petitions over this period has fluctuated but has generally declined since 2015, and none were filed in August or September 2017. USPTO has taken several steps to ensure the timeliness of trial decisions, review past decisions, and engage with stakeholders to improve proceedings under the program: Timeliness: USPTO regularly informs relevant parties about paperwork requirements and due dates throughout trials. According to program data, as of September 2017, all 181 completed trials were completed within statutorily required time frames. Decision review: USPTO has taken several steps to review its decisions and has monitored the rate at which the Court of Appeals for the Federal Circuit affirms or reverses them. However, USPTO does not have guidance, such as documented procedures, for reviewing trial decisions, or the processes leading to decisions, for consistency. Without guidance, such as documented procedures, USPTO cannot fully ensure that it is meeting its objective of ensuring consistency of decisions. Stakeholder engagement: USPTO judges have engaged with stakeholders by participating in public roundtables and webinars, and attending judicial conferences, among other things. Stakeholders GAO interviewed generally agreed that the CBM program has reduced lawsuits involving business method patents in the federal courts. While many stakeholders favored maintaining aspects of the program, there was not strong consensus among stakeholders for how future trials should be designed.