An Experimental Analysis on Evaluating Patent Citations (2024)

Abstract: The patent citation count is a good indicator of patent quality. This often generates monetary value for the inventors and organizations. However, the factors that influence a patent receiving high citations over the year are still not well understood. With the patents over the past two decades, we study the problem of patent citation prediction and formulate this as a binary classification problem. We create a semantic graph of patents based on their semantic similarities, enabling the use of Graph Neural Network (GNN)-based approaches for predicting citations. Our experimental results demonstrate the effectiveness of our GNN-based methods when applied to the semantic graph, showing that they can accurately predict patent citations using only patent text. More specifically, these methods produce up to 94% recall for patents with high citations and outperform existing baselines. Furthermore, we leverage this constructed graph to gain insights and explanations for the predictions made by the GNNs.

## introduction & related work patents play a pivotal role in driving innovation and fostering economic growth. they provide a legal framework that allows inventors (e.g., companies, researchers) exclusive rights to their creations for a specified period, typically 20 years @xcite @xcite . this exclusivity motivates the inventors and the businesses to invest in research and development, as they can benefit from their innovations. patent citations are important in the context of intellectual property (ip) and patent valuations and serve multiple important roles for patent examiners and applicants. firstly, they aid patent examiners in assessing an invention's novelty and nonobviousness for granting patents to genuinely innovative creations. secondly, they assist inventors by revealing the technological landscape and help them to refine claims and avoid any patent conflicts. thirdly, patent citations play a significant role in assessing the value of patent portfolios, with more citations often signifying greater influence in specific industries. further, researchers employ them to track tech trends and policy impact. several studies have analyzed patent value through the forward citations @xcite and assessed economic value of patents @xcite . previous research endeavors have explored broader patterns of knowledge transfer @xcite through patent citations such as interactions between academia and industry via citations between academic papers and patents @xcite . one of the relevant work involves prediction of patent value dependent on citation count from the text @xcite with regression. however, we differ in multiple ways: our study formulates a classification task, construct a semantic-based network, uses graph neural network (gnn)-based methods, and generates explanations. in this paper, we perform an extensive empirical study on the power of patent text to predict citations. our major contributions as follows. problem and data. we study the problem of patent citation prediction as a binary classification problem. our study includes granted patents over last two decades and provides descriptive analyses on the meta-data of the patents in three major classes. method. we construct a patent semantic graph from the patent similarities and use graph neural network (gnn)-based methods for citation prediction. our empirical evaluations show that the gnn-based methods can predict patent citations only using the patent text with high quality. explanation. the constructed graph combined with an explanation technique are used to get insights of the predictions of the gnns. note that we have added more details for next sections in the appendix along with background, related work, and additional experiments. ## problem definition and data we formulate the problem of patent citation prediction as a binary classification task where we classify the patents as highly cited or low cited. let @xmath0, p 2 , • • • , p m } be the set of m patents. as the patent citations vary over years, we use the count of citations obtained by a patent after d years from the year of being granted. we denote the citation of the patent p i after d years as c i d . in the experiments we use @xmath1, 5, and 10 years and use these to generate different labels and thus they generate different datasets. our aim is to measure the impact of a patent by using the citations of the patent. we focus on predicting whether a particular patent will be highly cited (positive, denoted by 1) or low cited (negative, denoted by 0) at the time of its granting year by using the text-based information from the patent itself. the decision on whether a patent belongs to a particular class (positive or negative) is based on the distributions of the citations. we set a threshold based on the distribution. let us assume the threshold is x-th percentile. thus, we define patent citation class as positive based on whether the citation count is higher than the value at the top x-th percentile. similarly, a patent belongs to a low cited class if the patent citation count is lower than the value at the bottom x-th percentile. definition 1 citation label: we define the label function y(c i d ) ∈ {0, 1} of a patent p i for the citations in next d years: other class labels. though the above definition produces citation labels, one could design the labels other ways. note that the above one produces an "easy to classify" dataset in the sense that the patent with high and low citations are well separated in the distribution. in the experiments, we explore other labeling settings. first, we define top x-th percentile as high, bottom x-th percentile as low and the rest as middle (we set @xmath2. as the main goal is to identify highquality or low-quality patents, we have divided the datasets and taken pair-wise classification in three our classification problem. we investigate the predictive power of the text in prediction of the quality of the patent, i.e., the patent citation count. to do so, we learn a prediction function f , where the features constructed from the patent text are given as input and the defined label y(c i d ) acts as the outcome variable. data. our study includes the granted patents from the united states patent and trademark office (uspto) foot_0 . the number of patents grow exponentially over the years. we have included recent patents over the last two decades from 2000 to 2022 for our analysis. our study focuses on citations which often depend on the area or topics of the invention, and thus, we consider on subcategories of patents. we consider patents under major (based on numbers) categories rather than all the patents. we follow the standard classification system for patents called the cpc categorization. we choose top three cpc classes in terms of the number of patents categorized in them. ## methods in patent citation prediction, there are two major challenges: (1) the texts in patents are not similar to the texts in research papers or news articles, (2) our aim to build models that are explainable, i.e, we can find the reasoning behind their predictions. text-based ai methods. modern ai tools have recently gained popularity in patent analysis @xcite . we use two methods to generate representations for the patent documents: doc2vec (lau and baldwin, 2016) and patent bert @xcite . these representations are used in combination with a multi-layered perceptron (mlp) for the classification tasks in the experiments. patentbert fine-tunes a pre-trained bert model with patent data and applies the model to the patent classification task. ## experiments we use three types of patent data from three major cpc (cooperative patent classification) classes: a61, h04, and g06. this results in nine separate datasets with three different periods of citation history from the year of 2000: (1) citation history for 3 years (3-years-history): patents published until 2019 as we count citation up to 2022; (2) citation history for 3 years (5-years-history): published until 2017. (3) citation history for 3 years (10-yearshistory): patents published until 2011. recent two years of data from each patent dataset (among the nine datasets above) are kept for testing. ## discussions we draw several key takes from the study. (1) text and network structure matter: graph-based ai models (gnns) can predict patent citation accurately only from the text of title, abstract, and claims. understanding the network structure of the patent landscape is also important. (2) explanation is the key: though several deep learning models have good predictive power, they might lack domain-specific explanations and the gnnbased explainers might be helpful. (3) recent data is important: the text from recent patents are more useful for citation prediction, thus, models should be mindful about the training data and possibly need re-training regularly. code and data are accessible at https://github.com/ robi56/patent_high_citation/ . ## ethical considerations in this work, we have built ai models based on textual information and patent semantic network to predict patent citations after the patents are granted. we do not foresee any ethical issues from our study. ## limitations this paper addresses a timely subject related to ai-based methods to predict patent citations. the dataset and the model used for this study are publicly available. while the paper shows the capability graph-based approaches towards patent citation prediction, one could further investigate the reasoning on patents getting high citations and build a few prototypes.