This section provides an overview of USPTO’s patent examination process, patent infringement litigation and challenges, and USPTO’s Enhanced Patent Quality Initiative. A list of our prior work related to patents and intellectual property is included at the end of this report. When USPTO receives a patent application, the agency assigns it to a division of patent examiners with relevant technology expertise called a technology center. There are 11 technology centers focusing on everything from biotechnology to mechanical inventions. After the application is assigned to a technology center, it is then assigned to an individual examiner who is responsible for the examination, or prosecution, of the application. Figure 1 shows the key steps in the patent prosecution process. The focus of patent examination is determining whether the invention in a patent application satisfies the statutory requirements for a patent, including that the invention be novel, useful, not obvious, and clearly described. Generally, prior patents, patent applications, or publications describing an invention, among other things, are known as prior art. During patent examination, the examiner, among other things, compares an application to the prior art to determine whether the invention is novel and not obvious. Finding prior art is the most time consuming part of patent examination, according to our report on prior art. Applicants are not required to search for prior art before submitting their application, although they are required to notify examiners of material prior art they know about. A patent application includes the “specification” and at least one “claim.” By statute, the specification is to contain “a written description of the invention, and of the manner and process of making it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains to make and use the invention.” The law requires further that the specification “shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor regards as the invention. A patent application’s claims define the legal boundaries of the invention for which patent protection is sought. USPTO’s data show that, as of April 2016, the average time between filing an application and an examiner’s initial decision on the application was about 16 months, and it takes an average of 26 months after an application is submitted for USPTO to complete the examination of an application. Due to the current inventory of applications awaiting examination, USPTO is not able to begin examining patent applications upon receiving them—as of April 2016, USPTO had a backlog of about 550,000 unexamined applications. Through the initial decision, or first action on the merits, examiners initially notify applicants about the patentability of their inventions. To avoid lengthy back and forth exchanges, the examiner is encouraged to identify all of the problems with the application under the patent statutes and rules during the first “office action”—a USPTO policy called “compact prosecution.” The applicant is able to respond to the examiner after the first office action, and interviews are often used so that the examiner and applicant can clarify their respective positions, as well as the scope of the claimed invention. Ultimately, and sometimes after many months, the examiner then issues a final rejection or allows the claims in the application. If applicants receive a final rejection, they may file a request for continued examination, which requires a new submission and the payment of additional fees, and the examiner will continue examining the application. There is no limit to the number of times an applicant may request continued examination for an application, but USPTO officials say it is rare to have three or more requests for continuing examination filed for a single application. Examination ends with an issued patent, or when the applicant abandons the application—there are no terminal rejections. There are often a dozen or more claims per patent, and they can often be difficult for a layperson to understand, according to legal researchers. For example, one claim for a cardboard coffee cup insulator begins by referring to “a recyclable, insulating beverage container holder, comprising a corrugated tubular member comprising cellulosic material and at least a first opening therein for receiving and retaining a beverage container.” A patent application’s claims can be written broadly or be more narrowly defined, according to legal researchers, and applicants can change the wording of claims—which can affect their scope—based on examiner feedback during examination. Examiners can suggest that applicants amend their claims by adding words to their claims to address statutory impediments to issuance. For example, adding the word “corrugated” to modify “tubular member” in the example claim above narrows the scope of the claimed invention. Companies often prefer broader patent claims that make it less likely a competitor would be able to make a small change to its invention to avoid infringement. Patents are a property right and their claims define their boundaries. In some cases, patent claims define the scope of the invention by encompassing an entire function––like sending an e-mail––rather than the specific means of performing that function. While “functional claiming” is permitted by statute, we reported in 2013 that patents that include functional claiming language were more likely to be unclear and to be disputed in court. For example, if the pencil was patented as a “mechanism for writing,” the owner of the patent could theoretically sue manufacturers of different technologies for infringement, including pens and markers. As of May 2015, USPTO had nearly 8,300 patent examiners across the eight technology centers that we reviewed. The agency uses the General Schedule (GS) classification system for patent examiners, whose levels range from GS-5 to GS-15. Examiners at the GS-14 level or above (44 percent of the examiners in the technology centers we reviewed) are referred to as “primary examiners” who may accept or reject a patent application without additional review. This level of authority is in contrast to junior examiners—most examiners below the GS-14 level— whose work must first be reviewed by a supervisory or primary patent examiner before it can be sent to the applicant. Examiners are rated based on their production, or the number of examination tasks they perform, among other factors. The number of examination tasks that an examiner is expected to perform is set based on the examiner’s technology area and experience level. USPTO allots more time to review applications to examiners that deal with more complex technologies. For example examiners working on artificial intelligence patent applications are given an average of about 31 hours to complete an examination, while those working on patent applications for exercise devices are given an average of about 17 hours. As examiners are promoted on the GS-scale, the average number of hours they are allotted to work on each application and the level of review from their supervisors declines. Primary examiners have the least amount of time to examine patent applications and the applications they review undergo the least amount of supervisory review, in part because their experience allows them to work more efficiently and effectively, according to USPTO officials. As examiners rise from junior to primary status, their examination time is roughly cut in half. According to USPTO officials, on average, examiners spend about 22 hours total on average on each application from start to final determination, with a low of about 11 hours on average for some primary examiners in the least complex technologies and a high of about 60 hours on average for an entry-level junior examiner in more complex technologies. A few studies have shown that there are differences in issued patents depending on how much time examiners are given to complete their examination. For example, an academic paper from 2012 found that more experienced examiners cite less prior art, are more likely to grant patents rather than reject them, and are more likely to grant patents without any preliminary rejections. Similarly, other researchers have found that examiners put less effort into searching for prior art when they are given less time to review an application. For example, the National Bureau of Economic Research published a paper in 2014 that found when examiners were allotted less time to conduct a patent examination, they were less likely to make time-intensive prior art rejections and more likely to grant a patent. When a patent right is not clearly defined, it can lead to boundary disputes, often in the form of infringement lawsuits. Although litigated patents are a small percentage of issued patents, low quality patents are more likely to be asserted in patent infringement lawsuits because, according to some economists, the less clear the claim boundaries are, the more likely that others will infringe the patent or will continue to infringe when confronted by the patent owner. Patent owners can bring infringement lawsuits against anyone who uses, makes, sells, offers to sell, or imports the patented invention without authorization. If the court finds that infringement has occurred, it must award the patent owner damages adequate to compensate for the infringement. During an infringement case, the accused infringer may seek to have the infringement lawsuit dismissed by showing, by clear and convincing evidence, that the patent at issue is invalid. However, because most of the roughly 4,000 patent infringement lawsuits filed each year settle before the court ever makes a determination of patent validity some patent owners asserting low quality patents may not be concerned with the risk the courts will invalidate their patents. When the courts do rule on validity, they generally invalidate almost half of the patents that are challenged, according to academic research. Accused infringers can also challenge a patent’s validity outside of an infringement lawsuit in administrative proceedings at USPTO’s Patent Trial and Appeal Board (PTAB). Patent challenges at PTAB are often initiated by individuals or firms that have been sued for infringing a particular patent, so the patents that appear in these proceedings are often the same ones that appear in patent infringement suits. The challengers in these proceedings seek to present evidence that shows that the patent claims should not have been granted because they failed to meet a statutory patentability requirement. The PTAB proceedings are a lower cost alternative to the federal courts where infringement suits are often very expensive. As of March 2016, there have been around 4,700 patent challenges filed with the PTAB since its inception in 2012 and about 60 percent of these challenged patents are related to computers and software. About 30 percent of the PTAB proceedings have reached a final decision, and nearly 75 percent of those final decisions have resulted in all of the challenged claims being held unpatentable. In February 2015, USPTO launched an Enhanced Patent Quality Initiative designed to improve the quality of patents. According to USPTO, it started its quality initiative because the agency had successfully reduced its backlog of patent applications, and had the financial resources to consider longer-term improvements to patent quality due to fee setting authority provided by the America Invents Act. As part of its initiative, the agency has taken a number of actions, including: Creating a new leadership role and Patent Quality Office: The agency created a new senior position for overseeing patent quality—the Deputy Commissioner for Patent Quality—in January 2015 to provide a dedicated focus on the agency’s patent quality efforts. Patent Quality Summit: USPTO held its first ever Patent Quality Summit in March 2015. The Summit was designed for the public, including internal and external stakeholders, to provide input to USPTO about patent quality, specifically how the agency could guarantee the most efficient process to review applications and to ensure the issuance of the highest quality patents, according to the USPTO Summit website. According to USPTO, the agency received 1,206 ideas for improvement for patent quality from all sources, including the Patent Quality Summit and examiner forums. Evolving programs of the Enhanced Patent Quality Initiative: USPTO established 11 initiatives based on feedback from internal and external stakeholders to help achieve its goals to enhance patent quality, according to its Enhanced Patent Quality Initiative website. These initiatives are in various stages, ranging from early development to having been completed, according to a senior USPTO official. Some of the evolving programs include: Clarity of the Record Pilot Program: USPTO began this pilot in February 2016 with about 130 examiners, and the pilot is expected to end in August 2016. This pilot seeks to develop best practices for examiners to enhance the clarity of all aspects of the prosecution record, and to study the effect of implementing these best practices during examination. Clarity and Correctness Data Capture (Master Review Form): This effort is expected to replace USPTO’s current quality assurance and supervisory approaches to reviewing examiners’ work and will allow the agency, for the first time, to collect consistent data across all the reviews. Quality assurance officials also told us that using the form would allow their office to have data on 50 percent more reviews this year than in the past, resulting in a total of about 12,000 reviews of examiners’ work this year compared with about 8,000 reviews per year in the past. USPTO expects to implement the form agency-wide in fiscal year 2017. In addition to the actions taken under its Enhanced Patent Quality Initiative, the USPTO’s Office of Patent Training develops and implements training for examiners on a variety of topics, with a focus on legal and policy matters. For example, the USPTO recently provided examiners with training on functional claiming. The number of federal district court filings of new patent infringement lawsuits has generally increased between 2007 and 2015, from more than 2,000 suits in 2007 to more than 5,000 suits in 2015 (see fig. 2). Because lawsuits can include multiple defendants, we also analyzed data on federal district court filings at the defendant level to account for lawsuits in which a patent was asserted against multiple defendants. Looking at the data in this way, we found that the number of defendants in new patent infringement suits filed in federal district courts increased from about 5,000 defendants in 2007 to more than 8,000 defendants in 2015, as shown in figure 2. According to some stakeholders with whom we spoke, the decreases in litigation that occurred in 2014, both in the number of suits and defendants, were likely due to a key Supreme Court decision in that year, Alice Corp. Pty. Ltd. v. CLS Bank Int’l. The Court in the Alice decision held that where a patent claim is based on an abstract idea, which is not patentable, merely using generic computer implementation does not transform that idea into a patent-eligible invention. This decision has limited the validity of what some stakeholders considered to be overly broad and low quality patents thus preventing them from being used to file infringement lawsuits. In addition, the ability of potential defendants in patent infringement suits to file inter partes challenges to the validity of a patent at the PTAB beginning in 2012 may have made some patent owners reluctant to bring infringement suits, which could have contributed to the decline in the number of suits according to some stakeholders we spoke with. Most patent infringement suits are filed in just a few of the 94 federal district courts, and these suits may generally be brought in any district in the country where the allegedly infringing products are sold. Patent infringement suits are increasingly being filed in the predominantly rural Eastern District of Texas (see fig. 3). In 2007, about 20 percent of all patent infringement defendants were named in cases filed in the Eastern District of Texas, and this percentage increased to almost 50 percent in 2015. Historically, the Eastern District of Texas has been attractive to patent owners filing infringement lawsuits because of the speed at which suits moved to trial, and the perception of plaintiff-friendly juries, as we found in 2013. In addition, according to one published paper we reviewed, judges in the Eastern District of Texas have implemented a number of court rules and practices to attract patent infringement suits to their district. For example, according to the paper and to some stakeholders we interviewed, judges in the Eastern District of Texas have not been suspending infringement lawsuits pending patent validity challenges at the PTAB, as some had anticipated. In addition, most recently, according to some stakeholders, some judges in Eastern Texas have been reluctant to dismiss infringement allegations at early stages of the litigation based on the Alice decision that calls into question the validity of certain types of software-related patents. Our analysis shows a sharp increase from 2014 to 2015 in the number of defendants that were accused of infringing software-related patents in suits filed in Eastern Texas, which have averaged about 85 percent of the defendants in that district each year between 2007 and 2015. None of the other top six district courts saw any large increases in patent infringement suits in 2015, including suits involving patents related to software or otherwise. This indicates that the increase in suits in Eastern Texas drove the overall increase in suits filed nationwide in 2015. The number of defendants in patent infringement litigation was relatively stable between 2007 and 2015, with the exception of lawsuits involving patents for computers and communications technologies (see fig. 4). This area includes technologies related to computer hardware and software. The percentage of defendants in lawsuits involving patents in this technology area increased from 38 percent in 2007 to 62 percent in 2015. Patents related to computer and communications technologies are easier to unintentionally infringe because they are more likely to be unclear and overly broad, according to some stakeholders we interviewed and some of the published research we reviewed. In addition, according to a 2003 Federal Trade Commission study, in industries such as computers and communications, firms need to avoid infringing dozens, hundreds, or even thousands of patents to produce just one commercial product. Therefore, it is particularly challenging to develop innovative new products where there are thousands of interrelated patents covering similar technologies making it nearly impossible to avoid infringement, according to some stakeholders we interviewed. These stakeholders also noted that unclear and overly broad patents—an indicator of lower quality—can harm innovation regardless of whether the patent owner files an infringement lawsuit because just the threat of an infringement suit can deter the development of new products. Further, the majority of defendants in patent infringement suits were involved in suits with software-related patents each year from 2009 through 2015 (see fig. 5). The number of defendants involved in infringement lawsuits where software-related patents were asserted generally increased through 2013, then dropped in 2014. As with computer and communications technologies, some stakeholders told us that software-related patents are easier to infringe because they also often have overly broad claims, an indicator of low quality patents. USPTO has taken actions to address patent quality—most notably through its Enhanced Patent Quality Initiative—but there are additional opportunities for USPTO to improve patent quality. Specifically, USPTO does not have a consistent definition of patent quality and has not fully developed specific performance measures to assess whether its efforts are affecting patent quality. Further, USPTO has not fully assessed whether the time it allots for examination and the monetary incentives it gives examiners for completing patent examinations faster affect patent quality. In addition, USPTO has limited data available on PTAB decisions. Finally, USPTO has not fully evaluated the effects of other policies and procedures on patent quality. USPTO does not currently have a consistent definition for patent quality, which may limit its ability to assess the effects of its examination policies and review processes—as well as its Enhanced Patent Quality Initiative— on patent quality. Several high level USPTO officials and the four supervisory patent examiners that we interviewed told us there is no consistent definition of patent quality at USPTO to guide the agency in its overall operations. One examiner wrote in our open-ended survey response that the USPTO appears to have no definition of patent quality and that without a working definition, management’s focus on patent quality is meaningless. Most of the stakeholders that we spoke with— including former high ranking USPTO officials, academics, and non- governmental organizations—said that it is important for USPTO to develop a consistent definition of patent quality. According to USPTO officials, one challenge in developing a consistent definition is that the patent community holds varying definitions of patent quality. For example, patent attorneys who defend companies against patent infringement generally tend to favor clearly defined patents that are easily understood in patent disputes, while patent owners tend to prefer less well defined patents that may offer broader coverage for their invention. USPTO officials offered a variety of definitions of patent quality. Several officials focused on validity, or patentability (i.e., meeting statutory requirements) and clarity, although we could not find this in agency guidance or documents as USPTO’s definition. Some officials also included aspects of patent examination, such as an application that requires a limited amount of time for review by examiners in their definition. While USPTO has found it difficult to clearly define patent quality, most of the stakeholders we spoke with told us that they would define patent quality as patent validity—that is, a quality patent would meet all the statutory requirements for patentability and would be upheld if challenged in a lawsuit or PTAB proceeding. According to federal standards for internal control, one important internal control activity is that management has established clear, consistent agency objectives that allow the agency to assess the risks the agency faces from external and internal sources. However, USPTO’s objectives may not be clear because it does not have a consistent definition of patent quality. Four supervisory patent examiners we interviewed told us that without a consistent definition of patent quality, USPTO is unable to standardize practices to improve patent quality. As a result, it is hard for USPTO to define, measure, and work toward quality goals, according to these supervisors. Moreover, without a definition for patent quality, USPTO is at risk of having agency officials work inconsistently or at cross-purposes in their attempts to fulfill the call to improve patent quality, based on individual understandings of what patent quality means to each staff person. Further, although USPTO is taking steps to improve patent quality metrics as part of the Enhanced Patent Quality Initiative, it has not established specific goals or performance measures related to its strategic goal to optimize patent quality and timeliness, which may limit its ability to assess potential effects of its efforts on patent quality. USPTO’s 2014-2018 strategic plan includes the goal to “optimize patent quality and timeliness,” but the patent quality objective does not include specific performance measures that fully assess progress towards the goal. For example, USPTO names seven objectives to achieve this goal, but six of the seven objectives focus on timeliness, customer service, and process or production goals rather than patent quality. For the one patent quality objective, USPTO cites improving patent quality data and maximizing the use of such data as two of its four performance measures (see fig. 6). The Government Performance and Results Modernization Act of 2010 (GPRAMA) requires, among other things, that agencies establish objective, quantifiable, and measurable performance goals, and establish performance indicators to measure progress toward each performance goal. Although USPTO is not required to comply with GPRAMA, we have previously reported the practices established by the law, such as establishing goals and performance indicators, can serve as a leading practice for organizational performance management at lower levels within federal agencies, such as individual programs or initiatives. However, USPTO has not established such goals and indicators because the office uses general terms to describe these two performance measures and does not include measurable, quantifiable goals or performance indicators on how the agency plans to define or collect patent quality data or how, specifically, it will maximize use of such data. Without such goals and indicators, USPTO cannot determine whether it is meeting its goal of enhancing patent quality. USPTO has not fully assessed the time allotted for patent examinations or its monetary incentive system, which may be at odds with patent quality. Patent examiners are rated annually on their production and docket management, among other elements. USPTO provides examiners with monetary incentives, or bonuses, for timeliness and production, but does not offer a bonus for producing high-quality work. Three USPTO officials told us that there are trade-offs between timeliness and patent quality, explaining that examiners cannot examine patents quickly and, at the same time, grant patents that are of the highest quality. One of these officials told us that the office’s focus on timeliness currently trumps high quality work at the agency, potentially increasing the tension between the goals of timeliness and quality. Time allotments and incentives can lead to pressure for examiners to complete their work quickly. Most of the stakeholders we spoke with told us that examiners’ time pressures are one of the central challenges for patent quality. The results of our survey of patent examiners confirm that examiners are experiencing time pressures. Specifically, on the basis of our survey, we estimate that, given a typical workload, about 70 percent of examiners have less time than needed to complete a thorough examination. In addition, we estimate that more than 70 percent of examiners worked voluntary or uncompensated overtime in the past 6 months to meet their minimum production goals (see fig. 7). Further, incentives appear to motivate examiners to complete their work quickly. For example, on the basis of our survey, we estimate that for 93 percent of examiners, receiving bonuses for achieving production goals motivates them to go above and beyond their base level of performance. Additionally, on the basis of our survey, we estimate that nearly 70 percent of examiners experience pressure to avoid time-consuming office actions. In addition, a few examiners we interviewed said that the system as currently designed incentivizes an examiner to issue a patent instead of issuing a final rejection, suggesting that when pressed for time examiners tend toward granting patents. For example, USPTO rules and procedures do not restrict the number of requests for continued examination an applicant may file, so issuing the patent is the only means USPTO has to end the examination process. This creates an environment where patents may be granted that do not fully meet patentability standards. In a 2015 study, USPTO economists found that patents examined by primary examiners had 26 percent better odds of appearing in a patent infringement lawsuit compared to similar patents examined by junior examiners, which suggests that primary examiners, despite their experience, may not have adequate time to ensure that the patents they issue are always high quality. USPTO officials acknowledged that this difference was likely due to the additional time and supervisory review that junior examiners receive. The precise effects of the time allotted for examinations and incentives on quality are unclear because USPTO has not fully analyzed the effects of current time allotments or incentives on an examiner’s ability to perform a thorough examination. Since creating the time allotments in the 1970s, USPTO adjusted the time allotted to examiners between fiscal years 2010 and 2012 and gave all patent examiners a total of 2.5 additional hours per application. According to federal standards for internal control, agencies should provide staff with the right structure, incentives, and responsibilities to make operational success possible. The right incentives and structure allow staff to be aligned with the agency’s objectives. Without analyzing the time and incentives needed for examiners to complete thorough examinations, USPTO cannot be assured that its current time allotments and incentives support the agency’s goal to optimize patent quality. The PTAB has been in operation since 2012 with hundreds of decisions made to date on challenges to existing patents; however, statistics kept by PTAB staff about the results of PTAB decisions are limited. For example, the data available as of March 2016, did not specify which precise claims in a patent were found to be unpatentable and why, or which sources of prior art were used in the proceeding, both of which are key data fields for potential analysis. Moreover, the data have not been widely shared within USPTO. According to a PTAB official familiar with the data, only a few USPTO staff in the Office of Patent Legal Administration have asked for and received the data that PTAB staff have compiled to date. Information on the results of the PTAB proceedings is not regularly provided to USPTO managers and supervisory patent examiners in the Technology Centers, according to PTAB staff. According to federal standards for internal control, information from internal and external sources should be obtained and provided to management as a part of the agency’s reporting on operational performance relative to established objectives. In addition, pertinent information should be identified, captured, and distributed in a form and time frame that permits people to perform their duties efficiently. However, USPTO managers and staff in the technology centers do not have data on trends in the outcomes of PTAB’s post-grant proceedings because USPTO has not systematically pulled information from PTAB decisions, has not widely shared PTAB information within the agency, and has not analyzed the data for trends in patent invalidations to identify whether additional training, guidance, or other actions are needed to address issues related to quality. Without a process for USPTO managers and staff to readily access the reasoning underlying PTAB’s invalidation of patents or information on possible trends that may exist with those problem patents, USPTO may be overlooking critical information on problems the PTAB has found with patents issued by the agency. As a result, USPTO may be missing opportunities for using this information to make decisions that could help to improve patent quality, such as improving guidance or providing additional training on potential patent quality issues, and USPTO’s examiners are unable to learn lessons from the PTAB decisions. USPTO has not fully evaluated the effects of other policies and procedures on patent quality, which may also affect their ability to issue high quality patents. Through interviews with USPTO officials and stakeholders and our examiner survey, we identified several policies and procedures that could affect patent quality: Compact prosecution: This USPTO policy encourages examiners to complete an examination within two office actions and to address all statutory issues of an application in the first office action. One examiner commented in our survey that compact prosecution compels examiners to guess what unclear claims mean in order to search for prior art. Further, based on our survey, we estimate that a change in policy that would not require examiners to address every issue in the first office action would help about half of USPTO examiners perform examinations more effectively. In addition, some stakeholders raised concerns about the effect of compact prosecution on patent quality, with one stakeholder emphasizing that the policy does not work well in an environment where patent applications are increasingly complex. Unlimited number of patent claims: Applicants are allowed to include any number of claims in a patent application. According to USPTO officials, applicants’ ability to file unlimited claims can have a negative effect on quality, because it is more difficult for examiners to fully review an application with numerous claims in the time allotted; this was also supported by our survey results. The 2015 study by USPTO economists also supported this finding; it found that patents that had more claims had better odds of showing up in patent infringement lawsuits. Requests for continued examination: Applicants are currently allowed to file an unlimited number of requests for continued examination, which is a request by an applicant to reopen examination of the patent application after the prosecution of the application has been closed. Applicants request continued examination most often after final rejection of an application, according to USPTO officials. Such requests provide applicants with virtually unlimited attempts to secure a patent, which is problematic for patent quality, according to some stakeholders. Some stakeholders also told us that such unlimited requests can wear down examiners, making them more likely to eventually grant the patent. While USPTO has reached out to patent applicants to learn why they were using such continued examinations, the number of requests for continued examination continues to pose a burden for the agency’s examiners. Federal standards for internal control direct agencies to comprehensively identify risks and consider all significant interactions between the entity and other parties. Once these risks have been identified, they should be evaluated for their potential effects, including the significance of the risks and the likelihood of their occurrence. USPTO officials acknowledged that some of the agency policies discussed above could affect patent quality; however, officials did not know the extent of the effects because USPTO has not done an evaluation to determine the potential effects. Without evaluating the effects of these policies on patent quality, USPTO is at risk of continuing practices that may adversely affect patent quality. Additionally, USPTO policies and procedures generally require clarity in issued patents. On the basis of our survey, we estimate that additional claim clarity requirements to applicants would help more than 80 percent of examiners do their jobs more effectively. We identified two areas where USPTO has encountered challenges with claim clarity in patent applications that can affect patent quality: (1) unclear terms, and (2) unclear or broadly worded claims including the use of functional claim language. First, patent applications that include unclear terms contribute to challenges with patent quality. On the basis of our survey, we estimate that 45 percent of examiners always or often encounter terms that are not well defined in the patent applications’ specifications. Most of the stakeholders we interviewed—including legal scholars and former high- ranking USPTO officials—as well as four supervisory patent examiners and the majority of examiners responding to our survey—indicated that requiring applicants to provide a glossary and define their terms would help to improve patent quality. Further, USPTO officials said that examiners and applicants that participated in USPTO’s Glossary Pilot Program generally indicated benefits to including a glossary and that the glossary improved claim clarity. Second, patent applications that include unclear or broadly worded claims, including those that use functional claim language, contribute to challenges with patent quality. For example, on the basis of our survey, we estimate that nearly 90 percent of examiners always or often encountered broadly worded claims in applications they reviewed, and for nearly two-thirds of examiners, applications with broadly worded claims make completing a thorough examination more difficult. In addition, dealing with functional claims, especially for software-related patents, can be time consuming and difficult especially if examiners are not aware of the applicant’s intent to use functional claiming language under section 112, according to a few examiners we interviewed. On the basis of our survey, we estimate that more than 40 percent of examiners experience pressure to avoid making rejections that relate to claim clarity (section 112). Also, according to our survey data, we estimate that for a smaller percentage of patent examiners (41%) correctly applying patentability standards for dealing with functional claims (section 112(f)) was very important compared to the percentage who reported adhering to novelty and nonobviousness standards (sections 102 and 103) was very important (81%). In a 2015 study, the USPTO economists found that patents with broader claims—patents with fewer words in their claims— had better odds of appearing in patent infringement lawsuits than similar patents. Further, that analysis found that patents that contained functional claim language had 40 percent better odds of appearing in patent infringement lawsuits than similar patents that did not contain this language. Some stakeholders we interviewed said requiring “claim charts” in patent applications could help improve patent quality. Claim charts use one column to present the claim and another column to present the limitations and boundaries of that claim (see table 1). These charts, which are commonly used in the federal courts and in PTAB’s post-grant proceedings, provide additional information on the boundaries of a patent claim. Similarly, according to some stakeholders, having applicants include a functional claim check box to indicate whether they were using functional claim language under Section 112(f) could help to improve patent quality. If the box is checked it indicates that the examiner should make sure that the functional claim is supported in the patent specification. By statute, an application for a patent must particularly point out and distinctly claim the subject matter of the invention. USPTO regulations require that the application include a description of the process of making and using the invention in such full, clear, concise, and exact terms as to enable a person skilled in the art to use and make the invention. However, examiners continue to encounter problems with patent application clarity because USPTO does not specifically require patent applicants to clearly define the terms used in their applications, provide additional means to clearly describe claim boundaries, or clearly identify when they are using functional claiming language. Without making use of tools to improve the clarity of patent applications, such as by having applicants include a glossary to define the terms used in the application, provide a claim chart, or indicate the use of functional claims through a checkbox, the agency is at risk of issuing unclear patents that may not comply with statutory requirements. The U.S. patent system plays a vital role in our nation’s economy by promoting innovation and supporting millions of jobs in innovation-rich sectors. However, the quality of patents issued by the USPTO has come under scrutiny in recent years, and our analysis suggests that some agency policies and practices may negatively affect the quality of the patents USPTO awards. It has taken several actions to elevate the importance of patent quality within the agency, but additional opportunities exist to improve the effectiveness of the agency’s patent quality efforts. USPTO does not have a consistent definition of patent quality that is clearly articulated in agency guidance or fully developed measurable goals and performance indicators to guide and evaluate work towards the agency’s quality goals. Without a consistent definition of patent quality, USPTO is at risk of having its staff work at cross purposes to improve patent quality based on their individual definitions of patent quality. Further, without improvements to measurable goals and performance indicators, USPTO is at risk of not being able to fully measure and capture key performance data on whether the agency is meeting its strategic goal to optimize patent quality. USPTO’s policies regarding the time allotted to complete patent application reviews and monetary incentives that are based on the quantity of the work examiners complete, not the quality of their work may negatively affect the quality of issued patents. However, USPTO has not analyzed its policies regarding examiner performance incentives and has assessed the time allotted for patent examination only minimally since the 1970s. Without analyzing whether time allotments to complete a thorough examination are sufficient, USPTO is at risk of issuing lower quality patents due to examiners’ not having enough time to complete their work. Our report on prior art searching recommended that USPTO reassess the time allotted to perform a thorough prior art search. Without analyzing the current incentive structure, USPTO cannot ensure that its incentives are aligned with high-quality work. Further, USPTO has limited data available on decisions from the PTAB to assess its patent quality initiatives. The PTAB has reviewed patents and invalidated some, or all, of the claims for hundreds of patents issued by the USPTO. However, the PTAB data and analysis of these decisions are limited in specificity as well as in distribution. Without identifying all of the data fields that would be useful to track, establishing a means for USPTO managers and examiners to easily access information—including information such as why claims were found to be unpatentable—and analyzing the data to identify potential trends, USPTO officials may be overlooking key information that could help them provide additional training or guidance or take other actions to address recurring issues. Further, other USPTO policies and procedures—namely, compact prosecution, unlimited numbers of patent claims, and requests for continued examination—may affect patent quality, but the agency has not evaluated the effects of these policies. Without evaluating the possible effects of these policies on patent quality, USPTO is at risk of continuing practices that could potentially affect patent quality. Finally, without requiring greater clarity in applications—through the use of glossaries, check boxes to signal functional claiming language, or claim charts—USPTO is at risk of issuing patents that are overly broad and not clearly worded—and may not comply with statutory requirements— thereby increasing the likelihood that the patent becomes the subject of litigation. We recommend that the Secretary of Commerce direct the Director of the USPTO to take the following seven actions to help improve patent quality: Develop a consistent definition of patent quality, and clearly articulate this definition in agency documents and other guidance. Further develop measurable, quantifiable goals and performance indicators related to patent quality as part of the agency’s strategic plan. Analyze the time examiners need to perform a thorough patent examination. This action could be taken in conjunction with the recommendation in our report on USPTO’s prior art search capabilities (GAO-16-479). Analyze how current performance incentives affect the extent to which examiners perform thorough examinations of patent applications. Establish a process to provide data on the results of the PTAB proceedings to managers and staff in the USPTO’s Technology Centers, and analyze PTAB data for trends in patent quality issues to identify whether additional training, guidance, or other actions are needed to address trends. Evaluate the effects of compact prosecution and other agency application and examination policies on patent quality. In doing so, USPTO should determine if any changes are needed to ensure that the policies are not adversely affecting patent quality. Consider whether to require patent applicants to include claim clarity tools—such as a glossary of terms, a check box to signal functional claim language, or claim charts—in each patent application. We provided a copy of our draft report to USPTO for review and comment. In its written comments, which are reproduced in appendix III, USPTO generally agreed with our findings, concurred with our recommendations, and provided information on steps officials plan to take to implement the recommendations. USPTO also provided additional technical comments, which we incorporated, as appropriate. USPTO stated in its response to the first recommendation that USPTO already has a consistent definition for patent quality, specifically that a quality patent is one that is correctly issued in compliance with all of the requirements of Title 35 as well as relevant case law at the time of issuance, which is consistent with how we define the term for this report. However, in our audit work, we did not find evidence that this definition was clearly articulated in agency documents and guidance or used in its performance indicators and goals. We revised this recommendation to clarify that USPTO should not only define the term, but also make this definition clear in relevant documents. In response to our second recommendation, USPTO said that it has taken some steps to update and improve its performance indicators and goals related to patent quality. In its technical comments, USPTO suggested that we revise the report to recommend that USPTO further develop its goals and performance indicators. We agree with this suggestion and made the change. As USPTO further develops its goals and performance indicators, we encourage the agency to more clearly link these goals and indicators to its definition of patent quality. In response to our seventh recommendation, USPTO said that, contrary to the draft report’s findings, USPTO’s initial conclusion was that a glossary did not make a meaningful difference in quality during the prosecution of an application, though the USPTO is still analyzing whether the use of a glossary has a long-term impact on a patent. In response to this comment, we revised the statement in the report to more closely align with the information that USPTO officials presented at its March 8, 2016, Patent Quality Chat on the issue. As agreed with your office, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies to the appropriate congressional committees, the Secretary of Commerce, the Director of the USPTO, the Commissioner for Patents, and other interested parties. In addition, the report will be available at no charge on the GAO website at http://www.gao.gov. If you or your staff members have any questions about this report, please contact me at (202) 512-3841 or ruscof@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix IV. To examine recent trends in patent infringement litigation, we obtained patent infringement litigation data from two companies—RPX Inc. and Lex Machina—that included all of the patent infringement lawsuits filed in all 94 federal district courts between 2007 and 2015. These data included information on the patents asserted in each suit, the defendants involved, and the federal district court where the suit was filed. We conducted data quality testing on the data from RPX and Lex Machina to look for missing or out of range values, interviewed relevant officials, and reviewed relevant documentation for the data and found the data to be sufficiently reliable to determine recent trends in patent infringement lawsuits. We also conducted 11 semi-structured interviews with stakeholders from technology companies, venture capital investors, and others knowledgeable about recent patent infringement litigation. In addition, we reviewed published literature on the patent system, including reports from patent researchers, the Federal Trade Commission, and the Congressional Budget Office. To examine what additional opportunities exist, if any, to improve patent quality, we reviewed relevant laws and USPTO documents and interviewed USPTO officials and representatives of the examiners’ union—the Patent Office Professional Association. We interviewed four supervisory patent examiners and six patent examiners from a variety of technology areas. We also conducted 11 semi-structured interviews with patent stakeholders who were knowledgeable about patent quality and USPTO, including legal scholars, former high-ranking USPTO officials, and representatives from public interest non-governmental organizations. Two of the 11 stakeholders we interviewed are currently serving in leadership roles as board members of the American Intellectual Property Law Association. In assessing USPTO’s efforts, we identified criteria in the federal standards for internal control and USPTO’s strategic plan. In addition, we conducted a web-based survey of a stratified random sample of 3,336 eligible USPTO patent examiners from across 8 of the 11 the technology-based subject matter groups (referred to as technology centers) into which USPTO examiners are divided. Fielded between August and November 2015, the survey was designed to collect information on USPTO’s approach to patent quality and how USPTO might improve its patent quality efforts. To identify our survey population, we obtained from USPTO a list of patent examiners as of May 2015. We excluded examiners from three technology centers, as follows: We excluded the Designs technology center because these examiners work on design patents instead of utility patents; design patents are outside the scope of this engagement and have different statutory and administrative requirements than utility patents. We excluded examiners who perform “re-examination” work and not initial patent examination. We excluded examiners in the patent training academy because these examiners are recent hires who are in a 12-month training program. We also excluded examiners employed at USPTO for less than one year. We then defined nine strata by technology center, with one technology center separated into two strata, as described in table 2. Specifically, the Transportation, Construction, Electronic Commerce, Agriculture, National Security and License & Review technology center includes a diverse set of technologies, including transportation, construction, agriculture, and business methods. In our review, we separated the art units—subunits of a technology center—focused on electronic commerce and business methods (collectively referred to as business methods) in light of recent legislation and court decisions related to business methods. This resulted in 9 strata with a target survey population totaling 7,825 eligible examiners. From this list, we drew our stratified random sample of 3,336 eligible USPTO patent examiners. We received responses from 2,669 eligible examiners for an 80 percent response rate. Because we used a probability procedure based on random selections, our sample is only one of a large number of samples that we might have drawn. Since each sample could have provided different estimates, we quantified the sampling error and express our confidence in the precision of our particular sample’s results at a 95 percent confidence interval. This is the interval that would contain the actual population value for 95 percent of the samples we could have drawn. We designed our sample to provide percentage estimates with 95 percent confidence intervals that are within upper and lower bounds of 5 percentage points, within each stratum. We oversampled based on an expected response rate of 70 percent; however, because we achieved a higher than expected response rate, the upper and lower bounds for survey results within each stratum are generally less than 5 percentage points. The only estimates for which the upper and lower bounds exceed 5 percentage points are certain results for the business methods stratum. In these instances, the upper and lower bounds are between 5 and 6 percentage points. In this report, our figures containing survey results show the upper and lower bounds for results at the 95 percent confidence interval. For other estimates in the report, we have not provided the upper and lower bounds in the text or tables; however, those details for all survey results are available in the e- supplement related to this report, GAO-16-478SP. The quality of survey data can also be affected by nonsampling error, which includes, for example, variations in how respondents interpret questions, respondents’ willingness to offer accurate responses, nonresponse errors, and data collection and processing errors. To minimize nonsampling error, we took several steps in developing the survey and in collecting and analyzing survey data. Specifically, in developing the survey, we worked with our survey professionals to, among other things, draft questions that were clear and unbiased. We pre-tested the survey in person with five USPTO staff: three examiners who are also representatives to the examiners’ union, a supervisory patent examiner, and a quality assurance specialist. We used these pre- tests to check that the questions were clear and unambiguous, used correct terminology, requested information that could be feasibly obtained, and were comprehensive and unbiased. We also obtained comments on the survey from USPTO management and leadership from the examiners’ union. In addition, we obtained a quality review by a separate GAO survey methodologist. Based on these activities, we made changes to the survey before administering it. Further, using a web-based survey and allowing examiners to enter their responses into an electronic instrument created an automatic record for each respondent. This eliminated the potential for errors that could have resulted if we had used a manual process to enter respondents’ data from paper surveys. In addition, to account for the complex sample design, we used survey software in our analyses to produce appropriate estimates and confidence intervals, and the programs we used to process and analyze the survey data were independently verified to ensure the accuracy of this work. To minimize nonresponse error, we made a variety of contacts with the sample of examiners during the survey, including follow-up e-mails to encourage responses. In addition, between October 20 and 23, 2015, we attempted to follow-up via telephone calls to all 1,102 examiners who had neither completed the survey nor told us that they were no longer examiners. We also analyzed nonresponse bias to (1) assess whether any factors were associated with examiners’ propensity to respond and (2) to allow our analysis of respondents to properly reflect the sampling universe of eligible examiners. To adjust the sampling weight for potential nonresponse bias, we used standard weighting class adjustments based on the sampling strata and the examiners’ years of experience at USPTO. In this report and in the related e-supplement at GAO-16-478SP, we present the survey results using the nonresponse adjusted weights, which are generalizable to the eligible population of examiners. We analyzed the responses to the survey for all examiners, as well as responses by technology center and by the General Schedule (GS) level of the examiners. We selected three categories of GS levels—less than GS-13, GS-13, and greater than GS-13—because examiners at these levels have different responsibilities and authorities when examining patent applications. Specifically, examiners above the GS-13 level may grant a patent or reject a patent application without additional review; examiners below the GS-13 level must have their actions reviewed and signed by a more senior examiner; and some GS-13 examiners are transitioning from one GS level to the other. For some other survey questions, we also reviewed examiners’ open- ended responses on selected topics. We selected those topics based on our interviews with experts and USPTO officials as well as our analysis of closed-ended survey responses. We selected the questions for which examiners’ responses most frequently included keywords we identified for each topic. An analyst conducted a keyword search of all responses to the selected open-ended questions and coded responses containing the keywords. A second analyst verified the initial analyst’s coding. Our report provides some examples of examiners’ comments based on this review. Examiners’ responses to open-ended questions are not generalizable to other examiners. In addition, because we did not conduct a systematic review of all open-ended responses to our survey, we do not report the exact number of examiners who provided responses on the topics we reviewed. In addition, we conducted statistical tests of association on the results of certain survey questions; all tests were independently verified to ensure their accuracy. All tests of association were carried out at the 5 percent level of significance and were Cochran-Mantel-Haenszel (CMH) Chi- square tests of general association. The testing was carried out in SUDAAN, which is statistical software appropriate for the analysis of survey data. The null hypothesis was that there is no association between the two tested variables. When the association between two variables, conditional on a third variable, is of interest, this relationship is referred to as the stratum-adjusted CMH test. The test statistic is Wald Chi-Square. We conducted this performance audit from November 2014 to June 2016 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Appendix II: Patent Examination Process at the U.S. Patent and Trademark Office (USPTO) (Corresponds to Fig. 1) This appendix provides details on steps in the patent examination process, including rollover information, depicted in figure 1. In addition to the contact named above, the following individuals made contributions to this report: Hilary Benedict (Assistant Director), Krista Breen Anderson, Richard Burkard, John Delicath, Cindy Gilbert, Shilpa Grover, Rob Letzler, Rebecca Makar, Rob Marek, Chris Murray, Eleni Orphanides, Shep Ryen, Kelly Rubin, Ardith Spence, Sara Sullivan, and Sonya Vartivarian. Intellectual Property: Assessing Factors That Affect Patent Infringement Litigation Could Help Improve Patent Quality. GAO-13-465. Washington, D.C.: Aug. 22, 2013. U.S. Patent and Trademark Office: Performance Management Processes. GAO-10-946R. Washington, D.C.: Sept. 24, 2010. Intellectual Property: Enhanced Planning by U.S. Personnel Overseas Could Strengthen Efforts. GAO-09-863. Washington, D.C.: Sept. 30, 2009. U.S. Patent and Trademark Office: Hiring Efforts Are Not Sufficient to Reduce the Patent Application Backlog GAO-08-527T. Washington, D.C.: Feb. 27, 2008. U.S. Patent And Trademark Office: Hiring Efforts Are Not Sufficient to Reduce the Patent Application Backlog. GAO-07-1102. (Washington, D.C.: Sept. 4, 2007. Intellectual Property: Improvements Needed to Better Manage Patent Office Automation and Address Workforce Challenges. GAO-05-1008T. Washington, D.C.: Sept. 8, 2005. Intellectual Property: Key Processes for Managing Patent Automation Strategy Need Strengthening. GAO-05-336. Washington, D.C.: June 17, 2005. Intellectual Property: USPTO Has Made Progress in Hiring Examiners, but Challenges to Retention Remain. GAO-05-720. Washington, D.C.: June 17, 2005.

Summary: Resolving disputes over patent infringement and validity in court often costs millions of dollars. Legal scholars and economists have raised concerns about an increase in the numbers of low quality patents—such as those that are unclear and overly broad—which may lead to an increase in patent infringement suits and can hinder innovation by blocking new ideas from entering the marketplace. GAO was asked to review issues related to patent quality. GAO examined (1) recent trends in patent infringement litigation and (2) what additional opportunities exist, if any, to improve patent quality. GAO reviewed relevant laws and agency documents; analyzed patent infringement litigation data from 2007 through 2015; conducted a survey of a generalizable sample of USPTO examiners; and interviewed officials from USPTO and knowledgeable stakeholders, including legal scholars, technology companies, and patent attorneys, among others. GAO found that district court filings of new patent infringement lawsuits increased from about 2,000 in 2007 to more than 5,000 in 2015, while the number of defendants named in these lawsuits increased from 5,000 to 8,000 over the same period. In 2007, about 20 percent of all defendants named in new patent infringement lawsuits were sued in the Eastern District of Texas, and by 2015 this had risen to almost 50 percent. According to stakeholders, patent infringement suits are increasingly being tried in the predominantly rural Eastern District of Texas, likely due to recent practices in that district that are favorable to the patent owners who bring these infringement suits. GAO also found that most patent suits involve software-related patents and computer and communications technologies. Several stakeholders told GAO that it is easy to unintentionally infringe on patents associated with these technologies because the patents can be unclear and overly broad, which several stakeholders believe is a characteristic of low patent quality. The U.S. Patent and Trademark Office (USPTO) has taken actions to address patent quality, most notably through its Enhanced Patent Quality Initiative, but there are additional opportunities for the agency to improve patent quality. For example, USPTO does not currently have a consistent definition for patent quality articulated in agency documents and guidance, which would be in line with federal internal-control standards and best practices for organizational performance. Most stakeholders GAO interviewed said they would define a quality patent as one that would meet the statutory requirements for novelty and clarity, among others, and would be upheld if challenged in a lawsuit or other proceeding. Without a consistent definition, USPTO is unable to fully measure progress toward meeting its patent quality goals. Additionally, USPTO has not fully assessed the effects of the time allotted for application examinations or monetary incentives for examiners on patent quality. Specifically, most stakeholders GAO interviewed said that time pressures on examiners are a central challenge for patent quality. Based on GAO's survey of patent examiners, GAO estimates that 70 percent of the population of examiners say they do not have enough time to complete a thorough examination given a typical workload. According to federal standards for internal control, agencies should provide staff with the right structure, incentives, and responsibilities to make operational success possible. Without assessing the effects of current incentives for examiners or the time allotted for examination, USPTO cannot be assured that its time allotments and incentives support the agency's patent quality goals. Finally, USPTO does not currently require applicants to define key terms or make use of additional tools to ensure patent clarity. Federal statutes require that patent applications use clear, concise, and exact terms. Based on a survey of patent examiners, GAO estimates that nearly 90 percent of examiners always or often encountered broadly worded patent applications, and nearly two-thirds of examiners said that this made it difficult to complete a thorough examination. Without making use of additional tools, such as a glossary of key terms, to improve the clarity of patent applications, USPTO is at risk of issuing patents that do not meet statutory requirements. GAO makes seven recommendations, including that USPTO more consistently define patent quality and articulate that definition in agency documents and guidance, reassess the time allotted for examination, analyze the effects of incentives on patent quality, and consider requiring applicants to use additional clarity tools. USPTO generally agreed with GAO's findings, concurred with the recommendations, and provided information on steps officials plan to take to implement the recommendations.