Automating Claim Construction in Patent Applications: The CMU mine Dataset (2021)

Abstract: Intellectual Property (IP) in the form of issued patents is a critical and very desirable element of innovation in high-tech. In this position paper, we explore the possibility of automating the legal task of Claim Construction in patent applications via Natural Language Processing (NLP) and Machine Learning (ML). To this end, we first create a large dataset known as CMUmine™and then demonstrate that, using NLP and ML techniques the Claim Construction in patent applications, a crucial legal task currently performed by IP attorneys, can be automated. To the best of our knowledge, this is the first public patent application dataset. Our results look very promising in automating the patent application process.

## introduction in the usa, european union (eu), and asia, most of the high-tech industries (semiconductors, wireless, internet, telecommunications, robotics, sensors, etc.) are characterized by the innovations they introduce and the inventions they make in a specific area of technology and these innovations are considered to be the intellectual property (ip) of these companies, a very important asset for any high-tech company. to protect their ip, high-tech companies file for patent applications to make it official that they own that specific idea and invention that could involve a new system or apparatus, new method, and new algorithms and/or software. in general, filing patents that describe new inventions is a lengthy, cumbersome, and very costly process since most high-tech companies have to hire law firms, litigation attorneys who specialize in ip, and technical professionals called patent "agents", or "patent engineers" for writing and filing such patent applications. after an inventor prepares a well-written document describing his/her invention, she/he submits this document to an ip attorney who takes this document that is known as "invention disclosure" and prepares a patent application that can be submitted in the us to the united states patent and trademark office (uspto). the main sections of a patent comprise the following sections: a key observation we make is that, if the invention disclosure is well prepared by the inventor, then the main contribution of the ip attorney to the patent application is to formulate the claims of the specific invention. in legal terms, this task is known as "claim construction". an ip attorney will not typically change the other parts of an invention (e.g., the specifications section of the invention); instead, he will formulate the main claims of the invention in the form of: (i) independent claims (ii) dependent claims and then append these claims to the end of the invention disclosure for the official submission of the patent application to the uspto. typically, patents in information technology have 3 independent claims and 5 or 6 dependent claims per independent claim that do depend on each of the 3 independent claims, thus resulting in a total of 20 or more claims in a patent application. in general, the independent claims concern the following aspects of an invention: 1) system or apparatus claim 2) method claim 3) software claim ## problem statement in this position paper, we explore the possibility of replacing the "human agents" with an automated solution (i.e., "the machine") in patent application process. in other words, we investigate whether the construction of the independent and dependent claims prepared by an ip attorney can be automated by using ai, natural language processing (nlp), and machine learning (ml). the key observation behind this is the underlying pattern in preparing the claims in a patent application: the ip attorney gets a well written invention disclosure from an inventor and, based on that and a brief conversation with the inventor, prepares the claims of a patent application. can this process be automated? our results suggest that the use of nlp and ml can indeed automate "claim construction" tasks and, therefore, the patent application process. as a proof-of-concept, among all the claims constructed by ip attorneys, this paper focuses on generating the first independent claim. the approach we pursue in this paper is to formulate the problem as a text summarization problem @xcite . ## related work text summarization aims to briefly summarize the key information of any longer input text. text summarization techniques using nlp has been successfully applied to various fields, including news @xcite , scientific papers @xcite and patents @xcite . specifically, in @xcite , the author models the text summarization task as follows: a granted patent's description is the input text, and its summary is regarded as the gold-standard summary. our task is similar to other text summarization tasks, i.e., summarizing the first independent claim from a longer invention disclosure. therefore, a viable approach that might work is to model our task as a text summarization task. ## data collection and dataset construction to meet our goal, we started by collecting a large dataset consisting of 300k patent applications, known as carnegie mellon university machine interpreted natural-language engineering (cmumine™). using this very large data set, dubbed cmumine™, of 317,356 previous us patent applications, we create a training set, a validation data set, and a test set that comprise 253,976, 31,736, and 31, 644 data points (i.e., previous patent applications), respectively. this is the first public and largest patent application dataset to the best of our knowledge. bigpatent @xcite is a well-known large text summarization dataset in patent domain, but it does not include patent claims and cannot be used for claim construction. ## approach as described in section 2, we formulate our problem as a text summarization task, so we use popular summarization systems: pointer generator (pg) @xcite and pegasus @xcite . we used the training data set to train our model, the validation data set to fine-tune our model in terms of hyper parameters and then evaluated the performance of our model in generating the first independent claim (fin) on the test set. the results obtained are compared with the "gold standard" that represent the versions of the same fin constructed by humans (i.e., ip attorneys). the obtained machine-generated results are compared in terms of rouge-1, rouge-2, and rouge-l @xcite scores based on the statistical distributions of the results obtained, i.e., the probability density functions (pdf's) of the results. it can be observed from ## discussion while the results reported look very promising, our current experiments have the following limitations: 1) we used abstractive summarization to generate the first independent claim which limits the input length. this might be solved by using a combination of extractive summarization, like heuristics, and abstractive summarization; 2) our results show that our generated claim is mainly a summary of what the inventor writes. however, the claims generated should also establish the novelty of the invention by looking at other patents and/or published papers in the public domain for similar inventions in the same space, instead of only focusing on one single patent application, which requires incorporating external knowledge; 3) using rouge score as the evaluation metric, which focuses on the syntax similarities between the generated claims and the gold standards, might not be sufficient to evaluate the quality of the generated claims. other aspects, such as semantic similarity, factuality, etc. need to be considered as well. we plan to address these problems in future work. ## conclusion our results suggest that claim construction and the patent application process can be largely automated in the future with the help of ai, natural language processing, and machine learning. this will have far-reaching consequences such as: we hope that our results will stimulate further research into using ai, natural language processing, and machine learning for automating the patent application process.