# Research on a Comprehensive Index System for Regional Patent Evaluation

Authors: Lixia Yu; Xinrui Zhan; Xingxin Zhao; Yunqing Liu
Journal: Mathematical Problems in Engineering
 (2022)

---

## Abstract

With the rapidly increasing number of patent applications in China, each applicant tends to accumulate the quantity of patent, but not the improvement of patent quality. Therefore, the State Intellectual Property Office (SIPO) has recently issued a package of notification, which aims to reverse the status quo. However, existing methods of regional patent evaluation still emphasize on quantity rather than quality indices, which leads to biased evaluation results. In this paper, we establish a comprehensive index system for regional patent evaluation. The indicators of the system consist of four dimensions, including quantity, quality, structure, and coordination. This aims to provide a path of high-quality development for regional patent work. Based on the patent data of each city in Sichuan Province in 2020, this paper validated the effectiveness of the comprehensive evaluation system and put forward countermeasures and suggestions.


---

## Body

## 1. Introduction

Since the Chinese government enacted the “Outline of the National Intellectual Property Strategy” in 2008, the number of patent applications has made a great increase [1]. Until 2020, the annual number of patent applications has consecutively ranked first in the world for 10 years. Though the number of patents increases, the patent quality worsens [2, 3]. According to the statistics, the majority of patent applications consists of utility model patents and design patents, which are in low quality compared with invention patents. The antecedents of the situation comprise two folds: first, the government departments attach great importance to the patent numbers [4], so they provide great number of subsidies to encourage patent applications [5, 6], while the quality of patents is neglected; second, in order to easily gain patent subsides from the government, enterprises tend to apply for utility model patents and design patents, which are easily granted but useless for innovation [7]. The existence of low-quality patents results in two negative effects [8, 9]. On the one hand, enterprises cannot yield economic profits from patent activities. On the other hand, the government cannot make a great progress in innovation though it has input lots of resources in patent activities.Nowadays, in the context of transition economy, the aim of economic development has transformed into high-quality growth. Hence, the government has paid more attention on the high-quality patents that can improve innovation ability. Correspondingly, the State Intellectual Property Office (SIPO) has issued policy documents, such as “Implementation Measures for Improving Patent Quality Projects” in 2017. Meanwhile, the SIPO urges regional governments that they should properly guide and support patent applicants to focus more on the quality of patents and establish a comprehensive index system to better accomplish the regional patent work. However, the existing index system used by regional governments has drawbacks in three aspects: first, the evaluation indicators are incomplete, hence cannot fully reflect the regional patent work [10, 11]; second, the index system is more quantity-oriented [12], thus leads to plenty of low-quality patents; third, the calculation of index system is simple so that cannot distinguish the importance of different types of patents.Based on the requirements of high-quality development of regional patent work, this research proposed a comprehensive index system for regional patent evaluation. First, our index system includes more evaluation indicators, which consist of four first-level indicators and 15 second-level indicators, so it makes the index system more complete. Second, we develop our index system from four dimensions, which comprise quantity, structure, quality, and coordination indices, which correspondingly make the index system more comprehensive. Finally, we use analytic hierarchy process (AHP) to build the index system, thus making the calculation process more scientific. Our research has the following contributions: on the one hand, this research has filled the gap of lacking research on the index system for regional patent evaluation. On the other hand, the newly developed index system can effectively and comprehensively evaluate the regional patent work and guide patent applicants to attach more importance on the patent quality rather than quantity.The paper proceeds as follows: Section2 introduces the extant patent evaluation research studies; Section 3 describes the principles and structure of our index system; Section 4 presents the application of the index system based on the regional-level patent data of Sichuan Province in China; Section 5 discusses implications and limitations of this evaluation system and concludes the paper.


## 2. Review of Patent Evaluation Research

There are a large number of previous studies regarding patent evaluation indicators. As innovation becomes more and more important in firm competence, the research of patent evaluation started to bloom. Scholars back in 1990s tended to establish the evaluation system using single indicators. However, these single-indicator-based methodologies are soon to be replaced by more comprehensive evaluation systems due to better data availability. Next, we will introduce some key indicators that are not only widely discussed in the extant literature but also the pillars of our evaluation system.First of all, the number of patent holding by firms can be thought of as an indicator for the successful outcomes from research and development process and reflects in the market value of the firm [13]. However, some scholars argued that solely using patent statistics can be a mirage appearing to provide a great number of objective and reliable proxies for innovation [14].Second, after a patent has been granted, the length of duration for the maintenance of patent rights can be thought of as an indicator of the business value of a patent. Lanjouw and Schankerman [15] showed that the period of patent maintenance is a powerful indicator of patent value. However, the duration of patent right maintenance can only be obtained after a long period. Therefore, other evaluation systems generally exclude this indicator due to timeliness.Third, some scholars evaluate the value of a patent through patent length and breadth. In other words, the more technology classes a patent can cover, the more valuable it will be. Kuhn and Thompson [16] proposed an easy-to-use measure of patent scope based on patent law and practices of patent attorneys. In the meantime, because the classification can be sometimes subjective, some scholars did not consider patent scope as a valuable indicator.Fourth, the number of citations of a patent is a widely recognized indicator of patent values. Harhoff et al. [17] believe that the number of patent citation can be used to assess the economic quality of a patent. However, for this indicator, the cumulative nature of forward citations needs a lengthy time period to be useful, and if there exists highly citing group for certain prior patents, there could be an upward bias.As mentioned above, the existing research on the patent evaluation using single indicator has been relatively mature. Afterwards, scholars pay more attention on comprehensive patent evaluation systems. In 1994, Narin as the founder of the CHI Research proposed the concept of patent measurement and then proposed the CHI patent evaluation system [18]. The system mainly consists of three basic indexes and 6 advanced indexes. Lanjouw and Schankerman [15] then established a comprehensive index for patent quality. This quality index is calculated using four indices, namely, the number of patent citations, claims, patent references, and patent family members. Also, they point out that adopting several indicators to evaluate patent quality will significantly reduce the measurement variance. Moreover, Mariani and Romanelli [19] learn from this model, employing the number of patent citations, claims, and patent family members to set an evaluation system for patent quality. The analysis index system of patent value launched by the China Technology Exchange in December 2012 comprehensively evaluates patent value from the legal status, technical intensity, and economic value. The index system includes three first-level indicators and 18 supporting indicators, and the Patent Value Degree (PVD) will be formed after the scores of experts are weighted.


## 3. Establishment of Principles and Comprehensive Index System for Regional Patent Evaluation

### 3.1. Principles of Evaluation

In order to establish a comprehensive index system for regional patent evaluation, we believe that quantity is the basis, quality is the key, and coordinated development is necessary. The following will explain each principle specifically.


#### 3.1.1. Quality-Oriented

The focus of the current patent management is taking measures to improve the patent quality under the new situation. The idea that “quantity is the key” should be reversed, and the quality should be made to be an important standard in regional patent evaluation. Emphasize the quality in the index system and highlight the central position of the patent quality in patent development for guiding innovators to pay more attention to the patent quality and advanced technology.


#### 3.1.2. Quantity-Considered

The number of patents is the basis of patent management and the most evident reflection of the achievement of regional patents. There may be a few low-value patents and some high-value patents and core patents among regional patents. Therefore, a comprehensive evaluation method to distinguish those high-value patents is also necessary. The total number of patents can reflect the overall innovation ability of a region and the government’s attitudes towards patent management. The number of high-value patents can reflect whether the development of the quantity coordinates with the quality and whether there is a deviation in the quantity. Taking quantity into account includes not only the assessment of the total number but also the regional layout of quantity, highlighting the number of high-value patents and core patents in the region.


#### 3.1.3. Systematic

The comprehensive strength of regional patents is a complex system composed of multiple levels and elements. In addition to the quantity and quality of patents in a region, the regional patent layout, the coordinative development of economy, society, and technology, and other factors should also be taken into consideration. Therefore, when we deliberate evaluation indicators, it is necessary to consider as many relevant key factors as possible. However, the evaluation index system should be logical and systematic rather than a simple combination of each index. Therefore, our study evaluates regional patents from quantity, structure, quality, and coordination, forming a clear analysis framework. These indicators are independent but interrelated, aiming to cover the most concerned key factors when evaluating patents.


#### 3.1.4. Feasibility

In theory, as many evaluation indicators as possible should be taken into consideration for establishing a comprehensive regional patent evaluation system, so as to assess the strength of regional patents comprehensively. However, in practice, the feasibility of indicators and the availability of data cannot be ignored, as well as the interference of human factors requires to be minimized. Besides, it would be better to employ objective quantitative indicators, which are available in the existing database or by simple searching procedures.


### 3.2. Establishment of Comprehensive Index System

Following the principles mentioned above, collecting the quantifiable indicators of patent evaluation at home and abroad, referring to the establishment of evaluation system for scientific and technological competitiveness, regional innovation capabilities, and scientific research performance of universities, and discussing opinions with experts, this study will divide the comprehensive index system into three levels. The first-level index leads to a composite index. The second-level index consists of four dimensions, namely, quantity, structure, quality, and coordination. The third-level supporting index includes 15 independent but interrelated evaluation indicators. Consequently, the comprehensive index system for regional patent is formed by 4 evaluation dimensions (the second-level index) and 15 evaluation indicators (the third-level index) through selecting indicators to estimate the patent composite index (the first-level index).The details of comprehensive index system for regional patents are shown in Table1.Table 1
Comprehensive index system for regional patents.
First-level indexSecond-level indexThird-level indexComposite indexQuantity index AAnnual applications A1PCT patent applications A2Invention patent granted A3Valid patents A4Structural index BThe proportion of invention patent applications to patent applications B1The proportion of invention patents granted to invention patent applications B2The proportion of valid invention patents to invention patents granted B3The proportion of PCT patent applications to invention patent applications B4Quality index CMaintenance rate C1Annual average citations of invention patents C2Average claims of invention patents C3Annual number of patent licensing, pledge, and transfer C4Coordination index DValid patents per 10,000 people D1Invention patents granted per ten billion of GDP D2Invention patents per 10,000 researchers D3


#### 3.2.1. Composite Index

As shown in Table1, the patent composite index is a criterion for the overall evaluation of regional patents. It includes both quality and quantity evaluation. It is formed on the basis of evaluating various indexes of regional patents. Therefore, it can objectively reflect whether the regional patent management meets the requirements of high-quality development and the needs of the country’s innovation-driven development.


#### 3.2.2. Quantity Index

Quantity index is an important second-level index; this is a quantitative characterization of patent management, which reflects the government’s attitudes towards patent management and the innovators’ attitudes towards patent application and maintenance. As for indicator selection, the total number of patent applications per year is used to reflect the overall volume of regional patents; the number of Patent Cooperation Treaty (PCT) applications and invention patents granted per year is regarded as the proxy for the number of high-value patents; the valid invention patent index can be used to evaluate the maintenance of regional patents.


#### 3.2.3. Structural Index

We also include the structure as a second-level index. The patent structural index is used to analyze the proportions of different types of patents in the region and evaluate whether there are structural problems in regional patent management. Invention patents, valid invention patents, and PCT patents can to some extent represent high-quality patents. They are selected to be the evaluation object of the structure index, reflecting whether the region is focused on the high-quality development of patent and whether there are a large number of low-quality patents in the patent application.


#### 3.2.4. Quality Index

Patent quality index is a comprehensive evaluation of regional patent quality, which evaluates patent quality from technology, law, and economy. In the selection of indicators, the patent maintenance rate is a more evident indicator to reflect the value of regional patents and the patent maintenance time is used as the index for evaluation. Moreover, the number of citations and claims are also relatively recognized as evaluation indicators of technological dimensions. Also, patent licensing, pledge, and transfer are the reflection of the activeness of patent economic activities.


#### 3.2.5. Coordination Index

The last second-level index is coordination. Regional patent management must be coordinated with the development of society, economy, and technology so as to make more contribution to regional economic development. The function of the patent coordination index is to reflect the coordinative degree of the development between patents and society, economy, and technology. The number of valid invention patents per 10,000 people, invention patents granted per ten billion of GDP, and invention patents per 10,000 researchers can comprehensively measure the quality of regional scientific research and the market application.


### 3.3. Calculation Method

In this study, Analytic Hierarchy Process (AHP) was employed to measure the weights of indicators at all levels, and the weight of each indicator in the evaluation index system is determined by its importance. The importance of each indicator is compared in pairs referring to the scores given by the experts and then design the judgment matrix. In this study, we select 10 experts that are from different areas but all are related to patent work. Those experts consist of three scholars from universities whose research interests are patent and innovation, four R&D engineers with master degrees from state-owned manufacturing firms, and three R&D engineers with master degrees from privately owned manufacturing firms. After calculating the weight vector, the accuracy and reliability are tested, checking whether they are logical. The application and analysis steps of the calculating method are as follows.


#### 3.3.1. Designing the Judgment Matrix

The indicators of the third-level index are compared in pairs referring to the scores given by the experts. The values are assigned by 1–9, and the pairwise comparison judgment matrix is formed as shown in Table2.Table 2
Judgment matrix of the second-level index.
The second-level indexQuantity indexStructural indexQuality indexCoordination indexQuantity index131/33Structural index1/311/31Quality index3313Coordination index1/311/31The judgment matrix can be simplified as follows:(1)A=131331311313313131131.


#### 3.3.2. Weight

(1)
CalculateMi, the product of each row in the judgment matrix:(2)M1=1×1×13×3=3,M2=1×1×13×3=0.1111,M3=3×3×1×3=27,M4=13×1×13×1=0.1111.(2)
Calculate the fourth root ofMi:(3)W1=34=1.3161,W2=0.11114=0.5774,W3=274=2.2795,W4=0.11114=0.5774.(3)
Normalization of vectorW=1.3161,0.5774,2.2795,0.5774T, that is, Wi=Wi/∑j=14Wj.(4)V1=0.2771,V2=0.1215,V3=0.4799,V4=0.1215.The weight vector isV=0.2771,0.1215,0.4799,0.1215T.


#### 3.3.3. Test of the Accuracy and Reliability

First, the consistency of the matrix is tested through the following steps. The maximum eigenvalueλmax=∑j=14AVj/4Vj=4.1545 and the consistency index CI=0.0515, so the average consistency index is CR=0.04153<0.1. It demonstrates that the second-level index judgment matrix A is consistent, and each weight is logical. Therefore, the weight of the second-level index is (0.2771, 0.1215, 0.4799, 0.1215).Similarly, the judgment matrix of each level is designed and calculated. The weight and composite weight of index at each level are shown in Table3.Table 3
Weight of comprehensive index system for regional patents.
The second-level indexWeightThe third-level indexWeightComposite weightQuantity index0.2771Patent applications0.12580.0349PCT patent applications0.07270.0201Invention patent granted0.55380.1534Valid patents0.24770.0686Structural index0.1215The proportion of invention patents0.38600.0468The proportion of valid patents0.29330.0356The proportion of PCT patents0.09780.0119The proportion of invention patent granted0.22290.0271Quality index0.4799Maintenance rate0.12150.0583Annual average citations of invention patents0.47990.2303Average claims of invention patents0.27710.1330Annual number of patent licensing, pledge, and transfer0.12150.0583Coordination index0.1215Valid patents per 10,000 people0.60000.0729Invention patents granted per ten billion of GDP0.20000.0243Invention patents per 10,000 researchers0.20000.0243


### 3.4. Calculation of Composite Index

Based on the comprehensive evaluation index system, by standardizing the indicators at three different levels, the patent composite index becomes a dimensionless value. The index value is multiplied by the index weight to calculate the quantity index, structural index, quality index, and coordination index. Similarly, composite index can also be calculated using the following method:(1)
The dimension of the third-level index is removed with the standard deviation method to calculate the evaluation value. The formula is as follows:(5)dij=Xij−MinXijMaxXij−MinXij×100,whereXij is the dimensional evaluation value of the third-level index, in which i denotes the second-level index and j denotes the third-level index. dij is the dimensionless value of the third-level index, and its value ranges from 0 to 100.(2)
The evaluation value of the second-level index is a weighted dimensionless value of the third-level index.(6)di=∑j=1niWjdij,whereWj is the weight of the third-level index and ni is the number of the third-level index under the second-level index i.(3)
The composite index is weighted and integrated by the second-level index, and its calculation formula is as follows:(7)d=∑i=1nWidi,whereWi is the weight of the second-level index, n is the total number of the second-level index, and di is the second-level index.


## 3.1. Principles of Evaluation

In order to establish a comprehensive index system for regional patent evaluation, we believe that quantity is the basis, quality is the key, and coordinated development is necessary. The following will explain each principle specifically.


### 3.1.1. Quality-Oriented

The focus of the current patent management is taking measures to improve the patent quality under the new situation. The idea that “quantity is the key” should be reversed, and the quality should be made to be an important standard in regional patent evaluation. Emphasize the quality in the index system and highlight the central position of the patent quality in patent development for guiding innovators to pay more attention to the patent quality and advanced technology.


### 3.1.2. Quantity-Considered

The number of patents is the basis of patent management and the most evident reflection of the achievement of regional patents. There may be a few low-value patents and some high-value patents and core patents among regional patents. Therefore, a comprehensive evaluation method to distinguish those high-value patents is also necessary. The total number of patents can reflect the overall innovation ability of a region and the government’s attitudes towards patent management. The number of high-value patents can reflect whether the development of the quantity coordinates with the quality and whether there is a deviation in the quantity. Taking quantity into account includes not only the assessment of the total number but also the regional layout of quantity, highlighting the number of high-value patents and core patents in the region.


### 3.1.3. Systematic

The comprehensive strength of regional patents is a complex system composed of multiple levels and elements. In addition to the quantity and quality of patents in a region, the regional patent layout, the coordinative development of economy, society, and technology, and other factors should also be taken into consideration. Therefore, when we deliberate evaluation indicators, it is necessary to consider as many relevant key factors as possible. However, the evaluation index system should be logical and systematic rather than a simple combination of each index. Therefore, our study evaluates regional patents from quantity, structure, quality, and coordination, forming a clear analysis framework. These indicators are independent but interrelated, aiming to cover the most concerned key factors when evaluating patents.


### 3.1.4. Feasibility

In theory, as many evaluation indicators as possible should be taken into consideration for establishing a comprehensive regional patent evaluation system, so as to assess the strength of regional patents comprehensively. However, in practice, the feasibility of indicators and the availability of data cannot be ignored, as well as the interference of human factors requires to be minimized. Besides, it would be better to employ objective quantitative indicators, which are available in the existing database or by simple searching procedures.


## 3.1.1. Quality-Oriented

The focus of the current patent management is taking measures to improve the patent quality under the new situation. The idea that “quantity is the key” should be reversed, and the quality should be made to be an important standard in regional patent evaluation. Emphasize the quality in the index system and highlight the central position of the patent quality in patent development for guiding innovators to pay more attention to the patent quality and advanced technology.


## 3.1.2. Quantity-Considered

The number of patents is the basis of patent management and the most evident reflection of the achievement of regional patents. There may be a few low-value patents and some high-value patents and core patents among regional patents. Therefore, a comprehensive evaluation method to distinguish those high-value patents is also necessary. The total number of patents can reflect the overall innovation ability of a region and the government’s attitudes towards patent management. The number of high-value patents can reflect whether the development of the quantity coordinates with the quality and whether there is a deviation in the quantity. Taking quantity into account includes not only the assessment of the total number but also the regional layout of quantity, highlighting the number of high-value patents and core patents in the region.


## 3.1.3. Systematic

The comprehensive strength of regional patents is a complex system composed of multiple levels and elements. In addition to the quantity and quality of patents in a region, the regional patent layout, the coordinative development of economy, society, and technology, and other factors should also be taken into consideration. Therefore, when we deliberate evaluation indicators, it is necessary to consider as many relevant key factors as possible. However, the evaluation index system should be logical and systematic rather than a simple combination of each index. Therefore, our study evaluates regional patents from quantity, structure, quality, and coordination, forming a clear analysis framework. These indicators are independent but interrelated, aiming to cover the most concerned key factors when evaluating patents.


## 3.1.4. Feasibility

In theory, as many evaluation indicators as possible should be taken into consideration for establishing a comprehensive regional patent evaluation system, so as to assess the strength of regional patents comprehensively. However, in practice, the feasibility of indicators and the availability of data cannot be ignored, as well as the interference of human factors requires to be minimized. Besides, it would be better to employ objective quantitative indicators, which are available in the existing database or by simple searching procedures.


## 3.2. Establishment of Comprehensive Index System

Following the principles mentioned above, collecting the quantifiable indicators of patent evaluation at home and abroad, referring to the establishment of evaluation system for scientific and technological competitiveness, regional innovation capabilities, and scientific research performance of universities, and discussing opinions with experts, this study will divide the comprehensive index system into three levels. The first-level index leads to a composite index. The second-level index consists of four dimensions, namely, quantity, structure, quality, and coordination. The third-level supporting index includes 15 independent but interrelated evaluation indicators. Consequently, the comprehensive index system for regional patent is formed by 4 evaluation dimensions (the second-level index) and 15 evaluation indicators (the third-level index) through selecting indicators to estimate the patent composite index (the first-level index).The details of comprehensive index system for regional patents are shown in Table1.Table 1
Comprehensive index system for regional patents.
First-level indexSecond-level indexThird-level indexComposite indexQuantity index AAnnual applications A1PCT patent applications A2Invention patent granted A3Valid patents A4Structural index BThe proportion of invention patent applications to patent applications B1The proportion of invention patents granted to invention patent applications B2The proportion of valid invention patents to invention patents granted B3The proportion of PCT patent applications to invention patent applications B4Quality index CMaintenance rate C1Annual average citations of invention patents C2Average claims of invention patents C3Annual number of patent licensing, pledge, and transfer C4Coordination index DValid patents per 10,000 people D1Invention patents granted per ten billion of GDP D2Invention patents per 10,000 researchers D3


### 3.2.1. Composite Index

As shown in Table1, the patent composite index is a criterion for the overall evaluation of regional patents. It includes both quality and quantity evaluation. It is formed on the basis of evaluating various indexes of regional patents. Therefore, it can objectively reflect whether the regional patent management meets the requirements of high-quality development and the needs of the country’s innovation-driven development.


### 3.2.2. Quantity Index

Quantity index is an important second-level index; this is a quantitative characterization of patent management, which reflects the government’s attitudes towards patent management and the innovators’ attitudes towards patent application and maintenance. As for indicator selection, the total number of patent applications per year is used to reflect the overall volume of regional patents; the number of Patent Cooperation Treaty (PCT) applications and invention patents granted per year is regarded as the proxy for the number of high-value patents; the valid invention patent index can be used to evaluate the maintenance of regional patents.


### 3.2.3. Structural Index

We also include the structure as a second-level index. The patent structural index is used to analyze the proportions of different types of patents in the region and evaluate whether there are structural problems in regional patent management. Invention patents, valid invention patents, and PCT patents can to some extent represent high-quality patents. They are selected to be the evaluation object of the structure index, reflecting whether the region is focused on the high-quality development of patent and whether there are a large number of low-quality patents in the patent application.


### 3.2.4. Quality Index

Patent quality index is a comprehensive evaluation of regional patent quality, which evaluates patent quality from technology, law, and economy. In the selection of indicators, the patent maintenance rate is a more evident indicator to reflect the value of regional patents and the patent maintenance time is used as the index for evaluation. Moreover, the number of citations and claims are also relatively recognized as evaluation indicators of technological dimensions. Also, patent licensing, pledge, and transfer are the reflection of the activeness of patent economic activities.


### 3.2.5. Coordination Index

The last second-level index is coordination. Regional patent management must be coordinated with the development of society, economy, and technology so as to make more contribution to regional economic development. The function of the patent coordination index is to reflect the coordinative degree of the development between patents and society, economy, and technology. The number of valid invention patents per 10,000 people, invention patents granted per ten billion of GDP, and invention patents per 10,000 researchers can comprehensively measure the quality of regional scientific research and the market application.


## 3.2.1. Composite Index

As shown in Table1, the patent composite index is a criterion for the overall evaluation of regional patents. It includes both quality and quantity evaluation. It is formed on the basis of evaluating various indexes of regional patents. Therefore, it can objectively reflect whether the regional patent management meets the requirements of high-quality development and the needs of the country’s innovation-driven development.


## 3.2.2. Quantity Index

Quantity index is an important second-level index; this is a quantitative characterization of patent management, which reflects the government’s attitudes towards patent management and the innovators’ attitudes towards patent application and maintenance. As for indicator selection, the total number of patent applications per year is used to reflect the overall volume of regional patents; the number of Patent Cooperation Treaty (PCT) applications and invention patents granted per year is regarded as the proxy for the number of high-value patents; the valid invention patent index can be used to evaluate the maintenance of regional patents.


## 3.2.3. Structural Index

We also include the structure as a second-level index. The patent structural index is used to analyze the proportions of different types of patents in the region and evaluate whether there are structural problems in regional patent management. Invention patents, valid invention patents, and PCT patents can to some extent represent high-quality patents. They are selected to be the evaluation object of the structure index, reflecting whether the region is focused on the high-quality development of patent and whether there are a large number of low-quality patents in the patent application.


## 3.2.4. Quality Index

Patent quality index is a comprehensive evaluation of regional patent quality, which evaluates patent quality from technology, law, and economy. In the selection of indicators, the patent maintenance rate is a more evident indicator to reflect the value of regional patents and the patent maintenance time is used as the index for evaluation. Moreover, the number of citations and claims are also relatively recognized as evaluation indicators of technological dimensions. Also, patent licensing, pledge, and transfer are the reflection of the activeness of patent economic activities.


## 3.2.5. Coordination Index

The last second-level index is coordination. Regional patent management must be coordinated with the development of society, economy, and technology so as to make more contribution to regional economic development. The function of the patent coordination index is to reflect the coordinative degree of the development between patents and society, economy, and technology. The number of valid invention patents per 10,000 people, invention patents granted per ten billion of GDP, and invention patents per 10,000 researchers can comprehensively measure the quality of regional scientific research and the market application.


## 3.3. Calculation Method

In this study, Analytic Hierarchy Process (AHP) was employed to measure the weights of indicators at all levels, and the weight of each indicator in the evaluation index system is determined by its importance. The importance of each indicator is compared in pairs referring to the scores given by the experts and then design the judgment matrix. In this study, we select 10 experts that are from different areas but all are related to patent work. Those experts consist of three scholars from universities whose research interests are patent and innovation, four R&D engineers with master degrees from state-owned manufacturing firms, and three R&D engineers with master degrees from privately owned manufacturing firms. After calculating the weight vector, the accuracy and reliability are tested, checking whether they are logical. The application and analysis steps of the calculating method are as follows.


### 3.3.1. Designing the Judgment Matrix

The indicators of the third-level index are compared in pairs referring to the scores given by the experts. The values are assigned by 1–9, and the pairwise comparison judgment matrix is formed as shown in Table2.Table 2
Judgment matrix of the second-level index.
The second-level indexQuantity indexStructural indexQuality indexCoordination indexQuantity index131/33Structural index1/311/31Quality index3313Coordination index1/311/31The judgment matrix can be simplified as follows:(1)A=131331311313313131131.


### 3.3.2. Weight

(1)
CalculateMi, the product of each row in the judgment matrix:(2)M1=1×1×13×3=3,M2=1×1×13×3=0.1111,M3=3×3×1×3=27,M4=13×1×13×1=0.1111.(2)
Calculate the fourth root ofMi:(3)W1=34=1.3161,W2=0.11114=0.5774,W3=274=2.2795,W4=0.11114=0.5774.(3)
Normalization of vectorW=1.3161,0.5774,2.2795,0.5774T, that is, Wi=Wi/∑j=14Wj.(4)V1=0.2771,V2=0.1215,V3=0.4799,V4=0.1215.The weight vector isV=0.2771,0.1215,0.4799,0.1215T.


### 3.3.3. Test of the Accuracy and Reliability

First, the consistency of the matrix is tested through the following steps. The maximum eigenvalueλmax=∑j=14AVj/4Vj=4.1545 and the consistency index CI=0.0515, so the average consistency index is CR=0.04153<0.1. It demonstrates that the second-level index judgment matrix A is consistent, and each weight is logical. Therefore, the weight of the second-level index is (0.2771, 0.1215, 0.4799, 0.1215).Similarly, the judgment matrix of each level is designed and calculated. The weight and composite weight of index at each level are shown in Table3.Table 3
Weight of comprehensive index system for regional patents.
The second-level indexWeightThe third-level indexWeightComposite weightQuantity index0.2771Patent applications0.12580.0349PCT patent applications0.07270.0201Invention patent granted0.55380.1534Valid patents0.24770.0686Structural index0.1215The proportion of invention patents0.38600.0468The proportion of valid patents0.29330.0356The proportion of PCT patents0.09780.0119The proportion of invention patent granted0.22290.0271Quality index0.4799Maintenance rate0.12150.0583Annual average citations of invention patents0.47990.2303Average claims of invention patents0.27710.1330Annual number of patent licensing, pledge, and transfer0.12150.0583Coordination index0.1215Valid patents per 10,000 people0.60000.0729Invention patents granted per ten billion of GDP0.20000.0243Invention patents per 10,000 researchers0.20000.0243


## 3.3.1. Designing the Judgment Matrix

The indicators of the third-level index are compared in pairs referring to the scores given by the experts. The values are assigned by 1–9, and the pairwise comparison judgment matrix is formed as shown in Table2.Table 2
Judgment matrix of the second-level index.
The second-level indexQuantity indexStructural indexQuality indexCoordination indexQuantity index131/33Structural index1/311/31Quality index3313Coordination index1/311/31The judgment matrix can be simplified as follows:(1)A=131331311313313131131.


## 3.3.2. Weight

(1)
CalculateMi, the product of each row in the judgment matrix:(2)M1=1×1×13×3=3,M2=1×1×13×3=0.1111,M3=3×3×1×3=27,M4=13×1×13×1=0.1111.(2)
Calculate the fourth root ofMi:(3)W1=34=1.3161,W2=0.11114=0.5774,W3=274=2.2795,W4=0.11114=0.5774.(3)
Normalization of vectorW=1.3161,0.5774,2.2795,0.5774T, that is, Wi=Wi/∑j=14Wj.(4)V1=0.2771,V2=0.1215,V3=0.4799,V4=0.1215.The weight vector isV=0.2771,0.1215,0.4799,0.1215T.


## 3.3.3. Test of the Accuracy and Reliability

First, the consistency of the matrix is tested through the following steps. The maximum eigenvalueλmax=∑j=14AVj/4Vj=4.1545 and the consistency index CI=0.0515, so the average consistency index is CR=0.04153<0.1. It demonstrates that the second-level index judgment matrix A is consistent, and each weight is logical. Therefore, the weight of the second-level index is (0.2771, 0.1215, 0.4799, 0.1215).Similarly, the judgment matrix of each level is designed and calculated. The weight and composite weight of index at each level are shown in Table3.Table 3
Weight of comprehensive index system for regional patents.
The second-level indexWeightThe third-level indexWeightComposite weightQuantity index0.2771Patent applications0.12580.0349PCT patent applications0.07270.0201Invention patent granted0.55380.1534Valid patents0.24770.0686Structural index0.1215The proportion of invention patents0.38600.0468The proportion of valid patents0.29330.0356The proportion of PCT patents0.09780.0119The proportion of invention patent granted0.22290.0271Quality index0.4799Maintenance rate0.12150.0583Annual average citations of invention patents0.47990.2303Average claims of invention patents0.27710.1330Annual number of patent licensing, pledge, and transfer0.12150.0583Coordination index0.1215Valid patents per 10,000 people0.60000.0729Invention patents granted per ten billion of GDP0.20000.0243Invention patents per 10,000 researchers0.20000.0243


## 3.4. Calculation of Composite Index

Based on the comprehensive evaluation index system, by standardizing the indicators at three different levels, the patent composite index becomes a dimensionless value. The index value is multiplied by the index weight to calculate the quantity index, structural index, quality index, and coordination index. Similarly, composite index can also be calculated using the following method:(1)
The dimension of the third-level index is removed with the standard deviation method to calculate the evaluation value. The formula is as follows:(5)dij=Xij−MinXijMaxXij−MinXij×100,whereXij is the dimensional evaluation value of the third-level index, in which i denotes the second-level index and j denotes the third-level index. dij is the dimensionless value of the third-level index, and its value ranges from 0 to 100.(2)
The evaluation value of the second-level index is a weighted dimensionless value of the third-level index.(6)di=∑j=1niWjdij,whereWj is the weight of the third-level index and ni is the number of the third-level index under the second-level index i.(3)
The composite index is weighted and integrated by the second-level index, and its calculation formula is as follows:(7)d=∑i=1nWidi,whereWi is the weight of the second-level index, n is the total number of the second-level index, and di is the second-level index.


## 4. Application of Comprehensive Evaluation Index System Taking Patent Data of Sichuan Province in 2020 as an Example

### 4.1. Calculation of the Patent Composite Index of Each City in Sichuan

To verify the validation of our comprehensive regional patent evaluation index system, we used the patent data of each city in Sichuan Province in 2020. Our data comprise two parts, including patent data and economic data. The patent data are retrieved from China National Intellectual Property Administration (CNIPA,https://www.cnipa.gov.cn/) and IncoPat (https://www.incopat.com/). CNIPA contains the most authoritative patent statistics and is well recognized in previous study [3]. IncoPat is a third-party patent service provider for complement. As for economic data related to population, economy growth, technology, and other aspects in the coordination index, they are retrieved from the national statistical yearbooks (http://www.stats.gov.cn/) and local government websites for public statistics of each city in Sichuan. In 2020, the number of provincial patent applications increases by 167676, reaching a year-on-year rise of 27.49%. There are 42676 invention patents, accounting for only 25%, while the rest are utility model patents and design patents. There are 108386 new patents granted, with a year-on-year growth of 32.08%, among which 14187 are invention patents. The number of provincial PCT patent applications increases by 530, and there are 8.41 invention patents owned per 10,000 people. By calculating the composite patent index of each city in 2020 (as shown in Table 4), the results can reveal underlying problems on patent quality behind the fast-developing figures.Table 4
The composite patent index of each city of Sichuan in 2020.
CityComposite indexQuantity indexStructural indexQuality indexCoordination indexChengdu64.33100.0075.6630.90100.00Panzhihua35.932.2487.5635.7762.89Nanchong34.833.3521.6265.472.16Mianyang29.8811.5181.5921.1454.57Aba29.410.3037.3849.979.17Zigong23.003.3152.8326.5025.22Guang’an22.262.0014.3041.751.61Liangshan21.491.0849.2728.4414.03Yibin21.285.4823.1734.146.10Deyang20.917.3239.8524.3720.05Suining20.822.7533.5031.0610.29Luzhou15.133.4428.9920.895.99Ganzi14.87030.9021.637.04Meishan14.622.9834.8816.5313.87Leshan13.653.0033.1515.8510.26Guangyuan13.591.1323.6721.671.02Bazhong13.571.2315.4622.386.09Dazhou12.942.7814.8521.401.74Yaan12.651.1738.7313.2710.75Neijiang11.252.6631.2812.008.20Ziyang11.201.1726.5813.639.58There is no patent in Ganzi in 2020, so the quantity index of Ganzi is 0.


### 4.2. Analysis of Evaluation Results

By comparing the composite index, there is significantly uneven development between the overall strength of patents in these cities. Chengdu has highest composite score, 64.33, while Ziyang has the lowest one, 11.20. Obviously, polarization is demonstrated. The average composite index of Sichuan Province is 21.79, with only 7 cities reaching the figure, so most cities need to improve their comprehensive strength of patents. Although the number of patent applications in Mianyang, Yibin, and Deyang soars in 2020, the deficiency of overall quality and the coordination with the rapid economic and social development of the city makes them lag slightly behind.By analyzing the second-level index, the development of each city is quite different. In terms of quantity index, Chengdu is prominent, with a huge number of patents and innovators, forming a “No. 1” in the figure. Because of the great gap between Chengdu and other cities, there may not be cities can surpass it in a short period. In terms of structural index, patent layout of Panzhihua is relatively scientific, focusing on the invention of high-value and high-quality patents. However, the patent structure of Bazhong and Dazhou is not as appropriate for the overhigh proportion of low-tech patents. In terms of quality index, the overall score is relatively lower than the other three indicators, indicating that the quality is the most serious problem in the patent development in Sichuan. Measures should be taken to improve the overall quality, such as issuing policy and assessment instructions. In terms of coordination index, the development of patents and cities in Chengdu, Panzhihua, and Mianyang are relatively coordinated. Also, the degree of coordination is much higher than that of other cities, becoming a height of coordinated development. The low coordination index of Guangyuan, Guang’an, and Dazhou shows that the local government does not pay enough attention to the development of intellectual property rights, so that the strategic position of patent work is not significant, the development of patents is obviously lagging behind the development of other cities, the relationship between economic growth and innovation activities is not close, and that the achievements of patents are inadequate.Based on the above analysis, it turns out that the main problem of patent work in the cities of Sichuan is “quantity first or quality first.” Taking Chengdu as an example, the comprehensive patent strength ranks first in these cities, and the quantitative index (100) takes the lead, but the quality index (30.90) lags behind. This reflects that its overall patent strength is “large but not strong” and the development of quantity and quality is imbalance. In the economically developed area of Sichuan Province, the rank of the quantity index is obviously higher than the quality index, reflecting that the local government is inclined to pursue the rapid increase in the number of patents rather than the quality in the patent work.


## 4.1. Calculation of the Patent Composite Index of Each City in Sichuan

To verify the validation of our comprehensive regional patent evaluation index system, we used the patent data of each city in Sichuan Province in 2020. Our data comprise two parts, including patent data and economic data. The patent data are retrieved from China National Intellectual Property Administration (CNIPA,https://www.cnipa.gov.cn/) and IncoPat (https://www.incopat.com/). CNIPA contains the most authoritative patent statistics and is well recognized in previous study [3]. IncoPat is a third-party patent service provider for complement. As for economic data related to population, economy growth, technology, and other aspects in the coordination index, they are retrieved from the national statistical yearbooks (http://www.stats.gov.cn/) and local government websites for public statistics of each city in Sichuan. In 2020, the number of provincial patent applications increases by 167676, reaching a year-on-year rise of 27.49%. There are 42676 invention patents, accounting for only 25%, while the rest are utility model patents and design patents. There are 108386 new patents granted, with a year-on-year growth of 32.08%, among which 14187 are invention patents. The number of provincial PCT patent applications increases by 530, and there are 8.41 invention patents owned per 10,000 people. By calculating the composite patent index of each city in 2020 (as shown in Table 4), the results can reveal underlying problems on patent quality behind the fast-developing figures.Table 4
The composite patent index of each city of Sichuan in 2020.
CityComposite indexQuantity indexStructural indexQuality indexCoordination indexChengdu64.33100.0075.6630.90100.00Panzhihua35.932.2487.5635.7762.89Nanchong34.833.3521.6265.472.16Mianyang29.8811.5181.5921.1454.57Aba29.410.3037.3849.979.17Zigong23.003.3152.8326.5025.22Guang’an22.262.0014.3041.751.61Liangshan21.491.0849.2728.4414.03Yibin21.285.4823.1734.146.10Deyang20.917.3239.8524.3720.05Suining20.822.7533.5031.0610.29Luzhou15.133.4428.9920.895.99Ganzi14.87030.9021.637.04Meishan14.622.9834.8816.5313.87Leshan13.653.0033.1515.8510.26Guangyuan13.591.1323.6721.671.02Bazhong13.571.2315.4622.386.09Dazhou12.942.7814.8521.401.74Yaan12.651.1738.7313.2710.75Neijiang11.252.6631.2812.008.20Ziyang11.201.1726.5813.639.58There is no patent in Ganzi in 2020, so the quantity index of Ganzi is 0.


## 4.2. Analysis of Evaluation Results

By comparing the composite index, there is significantly uneven development between the overall strength of patents in these cities. Chengdu has highest composite score, 64.33, while Ziyang has the lowest one, 11.20. Obviously, polarization is demonstrated. The average composite index of Sichuan Province is 21.79, with only 7 cities reaching the figure, so most cities need to improve their comprehensive strength of patents. Although the number of patent applications in Mianyang, Yibin, and Deyang soars in 2020, the deficiency of overall quality and the coordination with the rapid economic and social development of the city makes them lag slightly behind.By analyzing the second-level index, the development of each city is quite different. In terms of quantity index, Chengdu is prominent, with a huge number of patents and innovators, forming a “No. 1” in the figure. Because of the great gap between Chengdu and other cities, there may not be cities can surpass it in a short period. In terms of structural index, patent layout of Panzhihua is relatively scientific, focusing on the invention of high-value and high-quality patents. However, the patent structure of Bazhong and Dazhou is not as appropriate for the overhigh proportion of low-tech patents. In terms of quality index, the overall score is relatively lower than the other three indicators, indicating that the quality is the most serious problem in the patent development in Sichuan. Measures should be taken to improve the overall quality, such as issuing policy and assessment instructions. In terms of coordination index, the development of patents and cities in Chengdu, Panzhihua, and Mianyang are relatively coordinated. Also, the degree of coordination is much higher than that of other cities, becoming a height of coordinated development. The low coordination index of Guangyuan, Guang’an, and Dazhou shows that the local government does not pay enough attention to the development of intellectual property rights, so that the strategic position of patent work is not significant, the development of patents is obviously lagging behind the development of other cities, the relationship between economic growth and innovation activities is not close, and that the achievements of patents are inadequate.Based on the above analysis, it turns out that the main problem of patent work in the cities of Sichuan is “quantity first or quality first.” Taking Chengdu as an example, the comprehensive patent strength ranks first in these cities, and the quantitative index (100) takes the lead, but the quality index (30.90) lags behind. This reflects that its overall patent strength is “large but not strong” and the development of quantity and quality is imbalance. In the economically developed area of Sichuan Province, the rank of the quantity index is obviously higher than the quality index, reflecting that the local government is inclined to pursue the rapid increase in the number of patents rather than the quality in the patent work.


## 5. Conclusion

This paper establishes a comprehensive index system for regional patent evaluation. In order to evaluate regional patent comprehensively, we set 4 evaluation dimensions (the second-level index) and 15 evaluation indicators (the third-level index) in this system. The established principles of the index system include quality-oriented, quantity-considered, systematic, and feasibility. In China, the quantity of patents has increased rapidly while the quality has not kept pace.In our study, we emphasize the quality of patent in the evaluation system, which aims at reversing the present idea of “quantity-oriented” in patent work and regarding quality improvement as the core of patent development. We try to maintain objective and rigorous during the process of comprehensive regional patent evaluation, and most of the data are retrieved from public database. Therefore, the results should be relatively convincing. We use the patent data of Sichuan Province in China and calculate the scores for each index and each indicator. Accordingly, the composite index of each area of Sichuan Province is measured. By comparing the composite index, we can evaluate the comprehensive quality of regional patents. Moreover, we can provide advice to the development of regional patent through the index system.This comprehensive regional patent evaluation index system is currently built on the data that can be mostly available by public. Our study provides regional and local government an easy-to-use and low-cost method to evaluate their patent work. However, this index system also has its limitations. For instance, we cannot estimate the commercial value of a patent, which is an important quality indicator. Therefore, as the technology grows, we will be working on developing more scientific indicators using new technologies, such as big data, and machine learning to design and build a more inclusive and comprehensive evaluation system, which will better serve the national intellectual property strategy.



---

*Source: 1824245-2022-04-26.xml*