Patent- CR : A Dataset for Patent Claim Revision (2025)

Abstract: This paper presents Patent-CR, the first dataset created for the patent claim revision task in English. It includes both initial patent applications rejected by patent examiners and the final granted versions. Unlike normal text revision tasks that predominantly focus on enhancing sentence quality, such as grammar correction and coherence improvement, patent claim revision aims at ensuring the claims meet stringent legal criteria. These criteria are beyond novelty and inventiveness, including clarity of scope, technical accuracy, language precision, and legal robustness. We assess various large language models (LLMs) through professional human evaluation, including general LLMs with different sizes and architectures, text revision models, and domain-specific models. Our results indicate that LLMs often bring ineffective edits that deviate from the target revisions. In addition, domain-specific models and the method of fine-tuning show promising results. Notably, GPT-4 outperforms other tested LLMs, but further revisions are still necessary to reach the examination standard. Furthermore, we demonstrate the inconsistency between automated and human evaluation results, suggesting that GPT-4-based automated evaluation has the highest correlation with human judgment. This dataset, along with our preliminary empirical research, offers invaluable insights for further exploration in patent claim revision.

## introduction text revision aims to improve text quality, such as fixing grammar errors @xcite and enhancing sentence coherence @xcite . currently, datasets for this task are derived from scientific literature, wikipedia entries, and news articles @xcite . in this paper, we broaden the scope of text revision to encompass the domain of 1 https://github.com/scylj1/patent-cr patents, characterized by large-scale, complex, and precise textual data. the patent domain presents unique opportunities and challenges for the field of artificial intelligence (ai) and natural language processing (nlp) @xcite . patent claims are critical in a patent application document. as the legal centerpiece, they define the technical scope of the invention and ensure the patent can withstand legal scrutiny. we introduce the relevant background information of patents in appendix a. drafting and revising patent applications are both time-intensive and financially burdensome @xcite . research showed that large language models (llms) have the potential to generate high-quality patent claims but current performance is not yet satisfactory @xcite . to facilitate the automation of patent writing, we propose a new task, namely patent claim revision, aiming at improving the quality of patent claims to pass the legal scrutiny of patent offices. figure 1 illustrates an example of patent claim revision. by checking the dataset and consulting patent professionals, we have identified five different types of modifications between the draft and final versions. (1) content amendment: essential information missing in the draft is included to highlight novel features of the invention, and unnecessary or redundant information is removed, including overlaps with the prior art. (2) term consistency: technical terms are ensured to be consistent throughout the document. (3) language precision: grammatical errors are corrected, and word choice is refined for greater precision. (4) concision: some claims are merged with others for concision. (5) renumbering: the consolidation of claims necessitates adjustments in their numbering. our main contributions are detailed as follows: 1. we introduce a novel task namely patent claim revision and present the first dataset for research and evaluation, comprising 22,606 pairs of appli-1. a compressor section (100) for use in a gas turbine engine (20) comprising: a lower pressure compressor rotor (102); a high higher pressure compressor rotor (105) having a hub (109) and a plurality of blades extending radially outwardly from said hub (109) and an outer housing surrounding an outer periphery of said blades; and a tap (104) for tapping air at a radially outer first location in a duct position intermediate a the lower pressure compressor rotor (102) and the higher pressure compressor rotor (117)(105), passing the tapped air through a heat exchanger (106), and returning the tapped air to an outlet (110) at a second location which is radially inward of said first location, to provide cooling air adjacent to said hub (109) to cool said hub (109) along its length. , wherein said outlet (110) is also in said duct, and wherein a fan (108) drives air downstream of said heat exchanger (106) to said outlet (110). insertion deletion unchanged edits intention term consistency language precision content amendment renumbering merge for concision figure 1: an example of claim revision for patent ep3181869 cation and published claims originating from the same patent. 2. we conduct an empirical study with professional human assessments to evaluate different llms on this task. our findings reveal that most llms tend to simply input claims, leading to deviations from intended revisions. additionally, domain-specific models and fine-tuning demonstrate promising performance. moreover, despite the best performance of gpt-4 among all tested llms, its outputs remain substantially below the desired standard, highlighting the inherent complexity and challenge of accurately revising patent claims. 3. we assess the correlation between automated and human evaluations, revealing that gpt-4-based evaluations correlate most closely with human judgments. developing new automated evaluation metrics that better align with human assessments can be a promising future research direction. ## experiments we selected the patent claims in june 2024 as the test set and the remaining ones as the training set for fine-tuning or few-shot prompting. we introduce the experimental details in appendix c, including the prompts and environmental settings. ## results and discussion we report the empirical results of automated and human evaluations in ## conclusion we introduce the first dataset for english patent claim revision namely patent-cr, providing valuable resources for research and evaluation in this newly proposed task. our empirical study of var-ious cutting-edge llms and professional human evaluations reveal the inherent challenges of the task. most small-scale llms predominantly simplify inputs, deviating from the target purpose of claim refinement. law-specific models and the finetuning of general llms show promising performance. although gpt-4 outperforms other llms on this task, the output claims still need further refinement to meet stringent examination criteria, underscoring the task's complexity. additionally, we point out the inconsistency between automated and human evaluation results, suggesting that gpt-4-based evaluation has the highest correlation with human assessment. consequently, the patent claim revision task presents multiple challenges that necessitate resolution for advancements in this field.