Exploring Data-Centric Strategies for F rench Patent Classification: A Baseline and Comparisons (2023)

Abstract: This paper proposes a novel approach to French patent classification leveraging data-centric strategies. We compare different approaches for the two deepest levels of the IPC hierarchy: the IPC group and subgroups. Our experiments show that while simple ensemble strategies work for shallower levels, deeper levels require more sophisticated techniques such as data augmentation, clustering, and negative sampling. Our research highlights the importance of language-specific features and data-centric strategies for accurate and reliable French patent classification. It provides valuable insights and solutions for researchers and practitioners in the field of patent classification, advancing research in French patent classification.

## introduction a patent is a legal document that grants its holder the exclusive right to prevent others from making, using, selling, offering for sale, or importing the patented invention within a certain jurisdiction for a specific period of time. patent databases are valuable sources of information that reflect global innovations and technological developments. the number of patent applications has increased significantly over the past two decades, which can be attributed to various factors such as the increasing importance of technology in society, the role of patents in business valuation, and the globalization of commerce. due to the large volume of patents and patent documents, patent analysis and management have become complex and time-consuming. additionally, patents are granted within a specific domain, and patent classification is critical for patent scope and patent law. for the reasons mentioned above, automated patent classification systems have become essential for patent professionals to manage large collections of patents. the international patent classification (ipc) 1 is a widely used standard taxonomy for patent classification. it comprises a hierarchy of five levels, including section, class, subclass, group, and subgroup. the finest level, the subgroup, consists of over 70,000 leaf nodes. an example of an ipc code is shown in figure 1 . when submitting a patent application, the relevant technical fields are indicated by assigning one or multiple ipc codes at the subgroup level. this helps determine the workload of examiners responsible for assessing the application. however, the ipc undergoes regular minor updates and occasionally major restructuring. developing an automatic patent classification system is thus challenging due to the constantly evolving technological language and patent syntax, requiring ongoing efforts to address this moving target. in addition, the classification of patent text is a widely studied problem, particularly in languages with large patent markets, such as english, chinese, and japanese. however, due to variations in technical fields across different countries and markets, it can be challenging to directly transfer models or data trained on english patents to other languages. this problem is particularly challenging in the case of french patents, in particular presents unique challenges that require a sizable annotated corpus and experimentation with various classification models. despite the critical importance of french patents, few studies, such as @xcite , has focused on patent classification in french, and even those have only reported results on ipc subclass or group levels. to address this gap, this paper proposes a novel approach to french patent classification that builds upon different strategies aimed at addressing several unique challenges, including input length limitation, label imbalance, and data sparsity. our approach compares and evaluates different strategies for french patent classification at the two deepest levels of the ipc hierarchy : the ipc group and subgroups. to the best of our knowledge, this is the first comprehensive work specifically focused on patent classification at such deep levels for the french language. ## background a patent is a well-structured document that typically includes several sections, such as a title, abstract, background, brief summary of the invention, detailed description, one or more claims, drawings, and classification information. the international patent classification (ipc) is a widely used system for uniformly classifying the content of patents, with over 100 countries currently employing it. the ipc scheme, which was established by the world intellectual property organization (wipo) foot_0 in 1971, is hierarchical and serves as the preferred classification system for french patent classification. the ipc system consists of eight general categories of patents, represented by the section level of ipc. these categories are hierarchical and represent the broadest possible areas of technology, providing a starting point for patent classification. in practice, patent offices assign classification codes to patent applications to accurately describe the subject matter of the invention. they typically assign the most specific level of the international patent classification (ipc) to classify the documents in their database. by assigning a specific subgroup level of ipc, broader groups and higher levels of the ipc hierarchy that encompass it can be determined, as the ipc hierarchy is organized in a tree-like structure. this avoids any double classification of "parent" and "child" classes in the ipc hierarchy. in this work, we focus on exploring the effectiveness of classification methods on groups and subgroups separately. prior research on automated patent classification systems has typically relied on traditional algorithms and feature extraction methods @xcite @xcite @xcite . however, designing hand-crafted features can be time-consuming and lead to efficiency problems, making them difficult to apply to large patent collections. more recently, researchers have turned to deep learning techniques to leverage large-scale training data and generalize well to unseen data. for example, @xcite used lstm on 50 ipc subgroups, while deeppatent (li et al., 2018) employed a cnn with skip-gram word embeddings. @xcite trained fasttext word embeddings on full-text patents and then used gru models on top of these embeddings. pre-trained language models, such as ulmfit @xcite and @xcite , have also been fine-tuned for this specific task. comparing the performance of different pre-trained models for multi-label patent classification, xlnet @xcite was found to outperform other models @xcite . moreover, @xcite reduced the uncertainty of classification results through the fusion of multiple patent views. ensemble techniques have also been explored in patent classification. for instance, @xcite found that an ensemble of classifiers with separate inputs for title-abstract, claims, and description achieved the best performance. some researchers have explored mapping patent texts to international patent classification (ipc) codes using knn classification and hybrid methods that combine neural feature encoders with knn @xcite @xcite . another approach is to use a wide and deep (wnd) network @xcite to combine string-level similarity and semantic embeddings of patent text @xcite . the previous research on patent classification has primarily focused on flat classification, where the classification problem is considered at one specific shallow level of the hierarchy, such as the class or subclass level of the ipc. only a few studies have addressed the more detailed classification of patent data at the group and subgroup levels. for instance, (chen & chang, 2012) proposes a three-phase categorization algorithm using svm classifiers, @xcite formulates the hierarchical classification problem as a sequence generation task, and @xcite formulates the patent classification problem at deeper levels as an extreme multi-label text classification (xmtc) task. despite these efforts, many of these works still rely on feature extraction and model architecture improvements, which only partially address the challenges of accurate patent classification. in contrast, our work focuses on enhancing the quality of the training set and its input format for automatic patent classification, leveraging the same dataset proposed in @xcite , to improve model performance specifically at the most detailed levels of the ipc hierarchy. ## methodology the task can be framed as follows : given a patent document x, it is assigned with one or more ipc codes @xmath0, l 2 , . . . , l l }, where l represents the total number of predefined ipc codes at a specific level. our training set @xmath1, y i )|x @xmath2, y @xmath3, 1} l , @xmath4, 2, . . . , n } comprises instances x i and their corresponding labels y i , with the objective of learning a scoring function f . this function maps an input x i and a label l to a score f (x i , l) ∈ r and is optimized to maximize the score when y i,@xmath5.e., the label l is relevant to the instance x i ) and minimize the score when y i,@xmath6.e., the label l is irrelevant to the instance x i ). ## conclusion in this paper, we proposed and compared various data-centric approaches for french patent classification. through extensive experiments, we demonstrated that an ensemble strategy can significantly improve patent classification at shallower levels, such as the ipc group level. however, for deeper levels of classification, such as the ipc subgroup level, where data scarcity and long-tail label distribution are common problems, we recommend using data augmentation techniques, clustering, and negative sampling during the training process to improve model performance. our research makes a valuable contribution to the development of automated patent classification systems in the french language, and we hope that our findings will inspire further research in this area. in summary, our work highlights the potential of data-centric strategies to overcome the challenges associated with patent classification and lays the groundwork for future studies in this field.