Patent Response System Optimised for Faithfulness: Procedural Knowledge Embodiment with Knowledge Graph and Retrieval Augmented Generation (2024)

Abstract: A successful response to Office Action is crucial for an invention to obtain a patent. While previous attempts have applied generalised LLMs, such as GPT-4, in the response process, there remains significant room for improvement in generating faithful, unbiased, and practically valuable responses. To address this issue, we propose the Patent Response System Optimised for Faithfulness (PRO). PRO explicitly incorporates procedural knowledge used by patent agents during drafting arguments in response. This framework comprises several key components: (1) Our proposed PRLLM is a LLM tailored for patent responses, designed to have comprehensive patent domainspecific knowledge. (2) Our proposed PPNet encodes legal interpretations and relationships between technical components from judicial sources through a knowledge graph. (3) The augmented generation processes retrieve relevant information from both the patent text and PPNet to augment the PRLLM’s input and generate faithful responses. Results show that PRO significantly reduces unfaithfulness across six error types compared to several settings. For instance, PRO outperforms GPT-4 by an average of 39% in terms of faithfulness. This demonstrates the effectiveness of our domain-specific approach in improving the quality of automated patent responses.

heading: 1 Introduction
text: Large Language Models (LLMs), such as GPT4 (OpenAI, 2023) and LLaMa2 (Touvron et al., 2023), are deemed generalised and not domainspecific, posing challenges in the patent field. In the intellectual property field, patents filed with the United States Patent and Trademark Office (USPTO) are continuously evolving and growing, with new technologies and legal terms requiring complex analysis (USPTO, 2023). Recently, research has focused on developing or applying language models (LMs) and LLMs tailored for patent
language to address tasks such as patent drafting (Lee and Hsiang, 2020), prior art search (Lo et al., 2024), and semantic analysis (Chu et al., 2024).
Although these efforts have been made, LLMs have not significantly improved the Office Action (OA; e.g., rejection) and response (e.g., argument or amendment) process. This process involves detailed communication and extensive exchanges of technical and legal knowledge between examiners and patent agents to ensure the inventions’ novelty and non-obviousness. Chu et al. (2024) have started investigating the use of LMs/LLMs and recommender systems to automate patent responses. However, due to the concern of privacy, the distinctive nature of patent language, the uniqueness of each invention, and the intricacy of formulating responses, considerable improvements are still needed in patent response systems.
This leads us to our first research question: can we develop a domain-specific patent response LLM (PRLLM)? To investigate this, we constructed a dataset comprising patents and their corresponding OA-response histories over 10 years. This dataset also includes a wide range of types, domains, and tasks, ensuring comprehensive coverage. Incorporating previous data during training helps retain knowledge from earlier training phases, thus preventing the forgetting issue (Ibrahim et al., 2024). For the model, we selected LLaMa2 as the base model for continual pretraining among open-source LLMs. For supervised fine-tuning (SFT), we used paired OA-responses. The zeroshot results showed that while the model performs well in terms of formatting responses, identifying key legal and technical terms, it struggles with analysing examiners’ rejections (e.g., novelty or non-obviousness analysis), even when additional information is provided (see section 2.1 and section 5).
This raises another question: how can we enhance the faithfulness of PRLLM in developing
146
arguments? Empirically, patent agents utilise a series of procedural knowledge during response analysis. Upon receiving an OA, they first identify the points of contention (e.g., rejections). From there, they follow dual paths. The first path involves finding the core inventive concept related to the point of contention, which could be reflected in the patent and/or independent claims. This includes identifying the patent’s key features and its problem-solution. Agents then search for relevant past precedents to support their arguments (Garrod, 2010). The second path addresses rejections that the invention is similar to prior art. Agents analyse the relevant paragraphs in the patent that relate to the prior art, using them as the basis for their arguments. Combining these foundations, agents develop arguments and/or amendments to address the deficiency in the examiner’s Broadest Reasonable Interpretation (BRI) and/or in the patent’s claim. This type of procedural knowledge is not present in the previous LLMs, as it is implicit knowledge that agents use during the response process.
Hence, we propose a novel framework: Patent Response System Optimized for Faithfulness (PRO). This framework aims to explicitly incorporate procedural knowledge into the model. Specifically, the framework includes a patent precedents KG (PPNet) that represents the external precedents patent agents might refer to during developing ar-
guments. This KG characterises the relationships between invention technologies, not only in common or dictionary definitions but with legal interpretations that include judicial logic and specificity.
Additionally, the framework involves multiple Retrieval-Augmented Generation (RAG) processes, where retrievers use points of contention or prior art to retrieve relevant information in the patent, and the generator uses PPNet/PRLLM-retrieved results to produce key features, problem-solution statements, and the resulting response. Experimental results demonstrate that this framework significantly reduces unfaithfulness compared to baselines.
We make several key contributions:
• We pioneered the development of a domainspecific patent response LLM (PRLLM).
• We are the first to introduce PPNet, a KG of patent precedents. The KG serves as the foundation for retrieving relations between entities and is used in subsequent reasoning processes.
• We propose the framework PRO, which embodies the procedural knowledge used by patent agents in the response process. This framework combines PPNet and RAG with proposed PRLLM. This integration effectively enhances the faithfulness in PRLLM results.

heading: 2 Architecture Overview
text: Considering the domain-specific nature of patent responses, we first developed the patent response LLM. This model is designed to run locally for security reasons and is well-versed in patent language, various technical terms, relevant legal terminology, and the structure, format, and analysis required for responses. This LLM forms the core foundation of our entire technical architecture and can function as both a generator and a retriever within our framework (see section 3 for its training details).

heading: 2.1 Paradigm without Procedural Knowledge
text: As shown on the left side of fig. 1, the most intuitive way to use PRLLM is through zero-shot application. When a patent agent encounters an OA, they can directly use PRLLM to generate the response content. This represents the simplest form of application.
A slightly more complex approach (see fig. 1 (x)) involves breaking down the information in the OA and identifying relevant details to add to the model input. Specifically, this involves extracting the examiner’s rejections, relevant prior art and patent paragraphs, and the key claims under dispute. Given the model’s window size limitations, these extracted details are token-optimised before being provided as input to PRLLM, resulting in a more precise response compared to the zero-shot method.
Both of these methods are direct applications, which we refer to as the paradigm without procedural knowledge. While this paradigm is simple, it lacks the integration of procedural knowledge crucial to the patent response process, potentially limiting its effectiveness.

heading: 2.2 Paradigm with Procedural Knowledge (PRO)
text: As shown on the right side of fig. 1, our PRO framework explicitly incorporates the procedural knowledge used by patent agents into the system. It consists of dual paths: PPNet path and prior art retrieval path.
For PPNet path (see fig. 1 (a)), we first use regular expressions to extract points of contention and the corresponding independent claims from the OA. Using this information, we perform RAG to identify key features and problem solutions. Specifically, we retrieve relevant texts in the patent using cosine similarities of dense vector representations derived from the PRLLM. During the generation
phase, our generator takes this textual information to output key features, including the relevant components (entities) and their relationships (relations), as well as the problem-solution of the patent.
We then use these extracted components and relationships to query our constructed PPNet. This KG helps to retrieve the legal implications of technical details within the patent. For example, if one queries "what is a gate above?" it might answer "a gate is above a layer" and "above means neither ’directly above’ nor simply ’at a higher place than’", providing precise legal interpretations.
For prior art retrieval path (see fig. 1 (b)), the objective is to utilise the examiner’s cited prior art paragraphs (which challenge the novelty and nonobviousness of the invention) to retrieve relevant paragraphs of the current patent application. Since examiners typically specify the locations and content of these prior art paragraphs, we can extract this information using regular expressions. After extracting the relevant prior art content, we apply the retrieval method identical to the first path, using PRLLM to identify similar paragraphs in the current patent application. These retrieved paragraphs are then re-ranked based on their importance, with the examiner’s most critical paragraph prioritised, followed by other similar passages.
This approach reflects one fact: While examiners often indicate the specific locations of contentious parts in the patent, our method not only relies on these key passages for argumentation but also uncovers additional details in the invention that the examiner may have overlooked. These overlooked details can be used to supplement and strengthen our response analysis.
Finally, the results from the two paths—the components and judicial rationales retrieved from PPNet, the problem-solution of the patent, and the key independent claims, along with the relevant passages from the invention—are combined with the relevant content in the OA to form the input for PRLLM.
Before this input is fed into the LLM, we perform a CoT process. This process is designed to determine the priority and functionality of each input and use reasoning prompting. Different inputs hold different levels of importance in constructing arguments and analyses. It is crucial for the LLM to understand the functionality and priority of these inputs to create a coherent and logical response. By structuring the inputs in this way and using reasoning prompts, PRLLM can generate responses with
higher faithfulness and accuracy.

heading: 3 PRLLM Training Details
text: We followed the approach outlined by Touvron et al. (2023) in training our PRLLM models. Using LLaMA2 as the base model, we trained models with parameters of 13 billion (13B) and 70 billion (70B), naming the series PRLLM-13B and PRLLM-70B respectively. The training process was divided into two main stages: continual pretraining and supervised fine-tuning (SFT).

heading: 3.1 Continual Pretraining
text: Data. To create an effective pretraining dataset, we ensured diversity and comprehensive coverage in our data. The patent domain encompasses extensive legal and engineering knowledge from various fields, necessitating a dataset that reflects this diversity.
First, our dataset includes patent documents and OA records, spanning from 2003 to 2022. This dataset comprises a total of 956,779 patents and 1,269,271 OA records from USPTO, accounting for 55.08% of the entire dataset. Second, we incorporated publicly available online resources, such as academic papers (12.64%), websites (11.24%), Wikis (9.28%), books (2.19%), exam databases and code repositories (2.07%), and news articles (2.02%). Lastly, the dataset includes some internal resources, such as judicial rulings (5.41%).
This comprehensive dataset design ensures that our Patent Response LLM has access to rich and diverse data during the pretraining phase. Leveraging data from various fields helps reduce potential biases in the model’s patent response process. Ibrahim et al. (2024) have shown that incorporating data from different domains in the pretraining phase can maintain the generalization capabilities of LLM models.
Training. We initiated pretraining using an optimized autoregressive transformer. We employed the LLaMA2 13B and 70B versions. The training was conducted on an A100 GPU cluster, utilizing the AdamW optimizer combined with BFloat16 mixed precision to ensure training stability. Ad-
ditionally, we implemented Cosine Learning Rate Scheduling for learning rate adjustments. Each training batch consisted of 4M tokens. To mitigate model performance regression, we extended the training context length from the original 4K to 16K (Xiong et al., 2023). Table 1 outlines the attributes and pretraining hyperparameters of the PRLLM models.

heading: 3.2 SFT
text: Data. During SFT, our data is divided into two parts. The first part, directly related to PRLLM, consists of paired OA-response datasets from 2023, totaling 10,000 instances. We denote this dataset as Dpr. The second dataset is a general dataset (Dg) comprising 20,000 instances, which were sampled from a variety of sources such as UltraChat (Ding et al., 2023), Databricks-dolly-15k (Conover et al., 2023), and the Guanaco Dataset (Dettmers et al., 2024). The final dataset (D) used for fine-tuning is the union of these two datasets, D = Dpr ∪ Dg.
Training. We merged all instances and outputs from dataset D. Each instance and its corresponding output were separated by a special token. This unified dataset was used to perform SFT on the two PRLLM models. Next, we omitted the loss calculation on tokens from user instructions and applied a weighted autoregressive objective (Wang et al., 2023). The loss function used in this training process is:
L(Θ) = Ex∼D[−α ∑
i∈O log p(xi | x̃; Θ)] (1)
where α is 1 if x is from Dpr and 0.15 if x is from Dg, O means output, Θ represents the model’s parameters, and x̃ = (x0, x1, . . . , xi−1) represents the tokenized input sequence. In a similar vein, we utilised a cosine learning rate scheduler with learning rate of 2× 10−5 and a batch size of 128. The models were fine-tuned over a total of 2 epochs.

heading: 4 PPNet: Construction & Evaluation

heading: 4.1 Building PPNet
text: Similar to constructing the Wikidata KG (Vrandečić and Krötzsch, 2014), we built PPNet for
patent responses argument foundation. PPNet sources include judicial relationships of components and relevant judgment contents such as Markman Hearings (Creel, 2013; Garrod, 2010). The construction process involves several steps: First, we performed data cleaning and annotation on the collected materials. Next, we carried out knowledge extraction, which includes Named Entity Recognition (NER), attribute extraction, and relation extraction. These steps rely not only on existing NLP techniques but also on manual annotation or verification by patent agents, attorneys, and engineers. Through these procedures, we extracted key information from the judgments and stored it in the knowledge graph.
As a result, PPNet can be represented as a heterogeneous KG consisting of triplets (head, relation, tail), denoted as G = (E ,R, T ), where E is the set of entities (e.g., components), R is the set of relations (e.g., verbs), and T is the set of triplets. In total, PPNet comprises 4 million entities, 403 types of relations, and over 7 billion triplets.

heading: 4.2 PPNet QA Pipeline
text: To handle complex question-answering tasks in Knowledge Graph Question Answering (KGQA), we adopted a method inspired by Sen et al. (2023). Our implementation for KGQA on PPNet is as follows:
We use a sequence-to-sequence model to predict the distribution of relations that need to be traced in PPNet. That is, the decoder predicts a relation distribution in PPNet, performing this process for up to m-hops. Each hop generates a relationship distribution, indicating which relation might be traced in that step.
Specifically, before the QA process, let three sparse linear matrices be head-to-triplet Mh, relation-to-triplet Mr, and tail-to-triplet Mt. We start with an initial query entity vector e ∈ RNE and a relation vector r ∈ RNR . The entity from the query is represented as a one-hot vector in the entity space, which is mapped to a triplet vector using Mh. For the relation, we use the relation vector predicted by the model and map it to a triplet vector through Mr. Hence, the first hop can be expressed as:
τ = Mhe⊙Mrr (2)
where ⊙ denotes element-wise multiplication. Then, using the tail-to-triplet matrix Mt, the
weighted triplet vector τ is mapped back to an entity vector:
e′ = MTt (τ) (3)
where e′ represents the begin of the second hop. At each hop, only the top k weighted triplets are retained, and these triplets are converted into natural language representations.

heading: 4.3 Experiments and Results
text: Dataset. To test the PPNet QA pipeline in answering patent judgment-related questions, we constructed a dataset for the following experiments. This dataset was collaboratively built by patent agents, attorneys, and engineers. The questions involve previous precedents, focusing particularly on technical components and their associations with others. For instance, a question might be, "What does a metal apparatus comprise?" with a possible answer being "copper".
The entire dataset consists of 4,730 questions (3,000 for the training set, 300 for the validation set, and 1,430 for the testing set). These questions are well-defined, with some involving multiple hops of reasoning to thoroughly test the capabilities and accuracy of the QA pipeline.
Experimental Setup. For the model, we selected several sequence-to-sequence models that have performed well in previous seminal work (Sen et al., 2023; Wu et al., 2023; Baek et al., 2023), including T0 models (Sanh et al., 2021), Flan-T5 models (Chung et al., 2024), and T5 models (Raffel et al., 2020). In our experimental setup, we set m = 5, meaning that the model can extract up to 5 triplets in PPNet. For metrics, we used Hit@1, Hit@3, and Hit@5 as evaluation metrics to measure the performance of the models.
Results. As shown in table 2, the T5-11B model demonstrated superior performance in the KGQA task on PPNet with Hit@1 at 86.39%, Hit@3 at 88.93%, and Hit@5 at 89.42%, followed by the
T0-3B model and the T5-3B model. This indicates the effectiveness of not only large model size but also model architecture in capturing and retrieving relevant triplets from the knowledge graph.

heading: 5 Evaluation on Generation

heading: 5.1 Unfaithfulness Error Taxonomy
text: To evaluate the faithfulness of PRO, we defined a taxonomy of errors (see Table 3) based on Kim et al.’s (2024) typology protocol. Our taxonomy includes six types of errors: Intrinsic Entity Error (IN), Extrinsic Entity Error (EN), Intrinsic Event Error (IV), Extrinsic Event Error (EV), Reasoning Coherence Error (RC), and Irrelevant Evidence Error (IE) (see appendix A for details).
We made specific modifications to Kim et al.’s (2024): Noun-Phrase Errors were consolidated into Entity Errors because, in patents, modifiers can change the meaning significantly. Overgeneralization Errors were merged into Irrelevant Evidence Errors, as both involve information that is not relevant to the current point of contention. These adjustments ensure the error taxonomy is more applicable to the context of patent responses.
Additionally, in patent responses, inventors typically prefer not to have their claims restricted in scope. Therefore, amendments are less desirable compared to arguments. Hence, we introduced a domain-specific metric to measure faithfulness: Recall of Argument (RA) and Precision of Argument (PA). In this context, a generated response judged as an argument (rather than an amendment) is considered true, and vice versa.

heading: 5.2 Experimental Setup
text: To assess the quality of generated responses, we employed human evaluation, recruiting a group
of experts in the patent field to evaluate the generated responses based on the six types of errors and whether the generated content was an argument or an amendment. A total of 4,153 generated responses to OAs from 2020-2022, which were unseen by PRLLM before, were evaluated with the ground-truths (GT) (see appendix B for details).
Our evaluation included several different settings. For the paradigm without procedural knowledge, native methods with only generators were used, including two setups: zero-shot and integrated external resources with reasoning (CoT). For the paradigm with procedural knowledge (PRO), this framework included multiple modules such as RAG in fig. 1 (a), retriever in fig. 1 (b), and generator.
In our experiments, we used different combinations of LLMs. For instance, we used GPT-4 for RAG and the retriever, and our PRLLM for the generator. However, Ding et al. (2024) indicate that using the same large language model for both the retriever and generator in a RAG system can be beneficial, as it ensures consistency in language understanding and generation and leverages shared internal representations and knowledge. Therefore, we focused more on using the same LLM across all three modules.

heading: 5.3 Results
text: According to table 3 the PRO framework using PRLLM-70B performed the best, achieving the lowest error rates across all settings. Compared to the closed-source state-of-the-art GPT-4, PRO showed significant improvements: IN had a 7.8% error rate (+41% improvement), EN had an 8.33% error rate (+35% improvement), IV had an 11.69% error rate (+29% improvement), EV had a 14.10% error rate (+37% improvement), RC had a 20.03%
error rate (+45% improvement), and IE had a 19.08% error rate (+43% improvement). On average, PRO outperformed GPT-4 by 39%, demonstrating that our domain-specific PRO framework with PRLLM is superior to generalised LLMs.
Several trends are also revealed: Zero-shot performance is inferior to CoT, and CoT is less effective than the PRO framework, indicating that the inclusion of procedural knowledge results in the most faithful responses. Additionally, extrinsic errors are generally common than intrinsic errors, particularly in larger models, suggesting larger models may introduce irrelevant external information. Lastly, our findings confirm that using mixed LLMs across different modules does not perform as well as using a consistent LLM throughout.
Regarding RA and PA, the results show that PRO consistently achieved the best performance across all settings, indicating high accuracy. Specifically, in CoT setup, especially within PRO, RA was greater than PA. This indicates that the system is more inclined to analyse and rebut the examiner’s opinions rather than directly amending the claims to limit the scope of the invention. This behavior aligns with the practical tendencies in the patent industry, where arguments are often preferred over amendments to avoid narrowing the claim scope.
For qualitative results on generated arguments, we refer to table 4. Across three main settings, the formatting of the responses shows no significant issues. In the zero-shot setting, the arguments primarily restate the content of the OA and merely mention a claim without providing analysis, indicated by gray text, showing an IE error. In the CoT setting, the arguments start to take shape but often repeat the examiner’s points and prematurely agree with them, which is not encouraged in patent responses. This is shown in red text, indicating an IV error. Additionally, parts of the argument analysis lack complete logical reasoning, shown in orange text, indicating an RC error. In the PRO setting, the generated arguments closely match the GT, both in format and completeness of reasoning. This indicates that the PRO framework effectively enhances the quality of patent responses, demonstrating its effectiveness in the domain.

heading: 6 Conclusion
text: We introduced a domain-specific patent response LLM (PRLLM) that can run locally to ensure privacy during patent responses, as well as a KG of
patent precedents (PPNet). Our proposed PRO framework explicitise the procedural knowledge used by patent agents, combining PPNet and RAG with PRLLM, significantly enhancing faithfulness across six error types compared to PRLLM alone and outperforming the state-of-the-art generalised LLM, GPT-4. Future research can focus on prompt tuning in CoT, addressing other aspects of patent responses beyond novelty and non-obviousness, and considering the history trajectory of OAs to further improve response effectiveness.

heading: A Typology
text: To evaluate the faithfulness of our work, we employed six error types to examine the quality of the generated responses. These error types are adapted from Kim et al.’s (2024) typology protocol to fit the practical scenarios of patent responses. Here is a simple, short, and hypothetical source for exemplary purposes:
Patent Application: The present invention pertains to a device, wherein said device comprises a wood layer positioned above two copper gates to enhance conductivity.
Office Action: The claimed invention lacks novelty because a prior art reference also discloses a layer positioned above a gate to enhance conductivity.
Below are the definitions of each error type along with examples relevant to patent responses.
A.1 Intrinsic Entity Error (IN) An Intrinsic Entity Error occurs when there is a misrepresentation of named entities, quantities, or other surface realizations from a given source. This type of error also includes the incorrect combination of modifiers meant for one entity with another entity.
Incorrect Argument: The present invention comprises a wood layer positioned above three wooden gates, which is not identical to the prior art.
Correct Argument: The present invention comprises a wood layer positioned above two copper gates, which is not identical to the prior art.
A.2 Extrinsic Entity Error (EN) An Extrinsic Entity Error occurs when new entities are introduced that were not present in the given source, or when modifiers that are not presented in the source are incorrectly combined with entities.
Incorrect Argument: The present invention includes a wood layer positioned above a gold circuit, which is not disclosed in the prior art.
Correct Argument: The present invention comprises a wood layer positioned above two copper gates, which is not disclosed in the prior art.
A.3 Intrinsic Event Error (IV) An Intrinsic Event Error occurs when events mentioned in the source are misrepresented, either through misunderstanding the event.
Incorrect Argument: The present invention describes the wood layer is placed beside the gates.
Correct Argument: The present invention describes the wood layer is positioned above the gates.
A.4 Extrinsic Event Error (EV)
An Extrinsic Event Error occurs when new events that are not present in the given source are introduced.
Incorrect Argument: The present invention includes a wood layer is used to store spiritual energy, which is not disclosed in the prior art.
Correct Argument: The present invention a wood layer is positioned above two copper gates to enhance conductivity, which is not disclosed in the prior art.
A.5 Reasoning Coherence Error (RC)
A Reasoning Coherence Error occurs when there are logical flaws in the flow of reasoning within the generated explanation, leading to a lack of coherence or weak support for the claim.
Incorrect Argument: The present invention is not identical to the prior art.
Correct Argument: The present invention is novel because the wood layer is positioned above two copper gates, which enhances conductivity, unlike the prior art that only discloses a single gate.
A.6 Irrelevant Evidence Error
An Irrelevant Evidence Error occurs when the explanation includes evidence that does not directly support the claim, or when it makes broad statements or conclusions that extend beyond the provided evidence.
Incorrect Argument: The present invention uses eco-friendly materials, which is not relevant to the enhancement of conductivity discussed in the prior art.
Correct Argument: That the wood layer is positioned above two copper gates to enhance conductivity is not disclosed in the prior art which discusses a single gate without mentioning such an arrangement.

heading: B Human Evaluation Procedure
text: To evaluate the effectiveness of each setting described in section 5, we employed a human evaluation method. We recruited a total of 331 experts in the patent field, including patent applicants, engineers, agents, scholars, and attorneys. Of the participants, 30.21% were female, and the median education level was a master’s degree. Each participant was randomly assigned a varying number of evaluation cases. Each case included the published and public versions of the patent, the corresponding OA, the actual response to the OA, and a response generated by our experimental setup. Additionally,
each participant received experimental instructions and an informed consent form.
After reading the informed consent form and the experimental instructions, participants were required to evaluate the generated response based on the six predefined error types, specifically focusing on the examiner’s rejections related to novelty (35 U.S.C. § 102) and non-obviousness (35 U.S.C. § 103). These error evaluations were multi-select.
Beyond the error evaluations, participants also had to determine whether the content of the response was more aligned with an amendment or an argument. They then judged the true response content similarly. Upon completing their tasks, participants received compensation in compliance with labor regulations.
In total, the participants effectively evaluated 4,153 OAs, encompassing approximately 11K points of contention related to novelty and nonobviousness. This evaluation process ensured a comprehensive assessment of the generated responses’ quality.

heading: Future Work
text: Future research can focus on prompt tuning in CoT, addressing other aspects of patent responses beyond novelty and non-obviousness, and considering the history trajectory of OAs to further improve response effectiveness