[image] [image] Ressource protégée - Archive ouverte HAL Accéder directement au contenu

*[image] [image]*

* [Documentation](#)

• FR
Français (FR)

• Anglais (EN)
Se connecter

- [HAL science ouverte](#)

* Recherche
Loading...

Recherche avancée

Information de documents

Titres

○ [Titres](#)

○ [Sous-titre](#)

○ Titre de l'ouvrage

○ <u>Titre du volume (Série)</u>

○ <u>Champ de recherche par défaut (multicritères) + texte intégral des PDF</u>

○ Résumé

○ Texte intégral indexé des documents PDF

○ Mots-clés

○ Type de document

○ Sous-type de document

○ <u>Tous les identifiants du document</u>

○ <u>Identifiant HAL du dépôt</u>

○ <u>Langue du document (texte)</u>

○ Pays (Texte)

○ Ville

○ <u>À paraître (true ou false)</u>

Ajouter

Auteur
<figure><img src="image_1.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
## Auteur (multicritères)

○ Auteur (multicritères)

○ Auteur : Nom complet

○ Auteur : Nom de famille

○ Auteur : Prénom

○ Auteur : Complément de nom, deuxième prénom

○ Auteur : Organisme payeur

○ Auteur : IdHal (chaîne de caractères)

○ Auteur : Fonction

○ Auteur : personID (entier)

○ Auteur : Identifiant de l'organisme payeur

○ Auteur : Identifiant de la structure

○ Directeur de thèse

○ Éditeur

○ Éditeur scientifique

○ Éditeur de la série

## Ajouter

### Structure

### Structure (multicritères)

○ Structure (multicritères)

○ Structure : Acronyme

○ Structure : Nom

○ Structure : Code

○ Structure : Pays

○ Structure : Type

○ Structure : État dans le référentiel

○ Structure : Identifiant HAL de la structure

○ Structure/équipe de recherche : Pays

○ Structure/regroupement d'équipes : Pays

○ Structure/laboratoire : Pays

○ Structure/regroupement de laboratoires : Pays

○ Structure/institution : Pays

○ Structure/regroupement d'institutions : Pays

## Ajouter

## Autres
Automatic TRIZ domain mapping

Guillaume Guarino¹ and Denis Cavallucci¹

¹ ICUBE/CSIP Team (UMR CNRS 7357) - INSA Strasbourg, France
firstname.surname@insa-strasbourg.fr

**Abstract.** The automatic analysis of patents is still one of the main challenges in R&D, particularly in terms of establishing automatic states of the art. Indeed, this is still mostly done manually, which is very time-consuming. The progress of artificial intelligence allows us to go a step further in the understanding of patents and in particular of the issues they address. In this paper we present an end-to-end tool that allows us to map the main trends in term of research directions in a sector in a few minutes from a simple keyword search. To do so, we will rely on TRIZ formalization with contradictions and evaluation parameters.

**Keywords:** Domain Mapping, TRIZ, Deep Learning.

# 1 Introduction

The identification of key issues in a field is very important to guide the research work of companies, especially when a company enters a new field of activity. With the technological challenges that lie ahead due to the energy transition, many companies will have to innovate in new areas. Reading patents does not provide a reliable mapping of a field, particularly because it is impossible for a team of engineers to read hundreds or even thousands of patents in a few days. In parallel, TRIZ theory allows problems to be formulated in a uniform manner across domains, in the form of contradictions. Progress in AI, particularly with deep neural networks, provides extremely powerful tools for analyzing textual content. Indeed, in an increasing number of NLP tasks, algorithms are actually becoming better than humans. It therefore seems possible to create a tool for fined-grained analyses of patents contents, regardless of the field, to provide a mapping of it in terms of research priorities. This is what we are going to present in this paper. From a simple search by domain, by keyword or even by applicant names we are able to provide a map in the form of a matrix highlighting the contradictions that are most dealt with by the patents. To build this map, an automatic extraction of contradictions resolved by each of the analyzed patents had to be set up. We will therefore start by presenting the contradiction extraction mechanism, then the construction of the map. Finally, we will show concrete examples of mappings.
Champ de recherche par défaut (multicritères)

○ <u>Champ de recherche par défaut (multicritères)</u>

○

○ <u>Revues (multicritères)</u>

○ <u>Revue : Éditeur</u>

○ <u>Revue : Titre abrégé</u>

○ <u>Revue : Titre</u>

○ <u>Revue : Identifiant interne</u>

○ <u>Revue : Date de début de publication</u>

○ <u>Revue : ISSN électronique</u>

○ <u>Revue : ISSN</u>

○ <u>Revue : Éditeur</u>

○ <u>Revue : Couleur dans SHERPA/RoMEO</u>

○ <u>Revue : État dans le référentiel</u>

○

○ <u>Colloque (multicritères)</u>

○ <u>Colloque : Titre</u>

○ <u>Colloque : Organisateur</u>

○ <u>Colloque : date de début (Année)</u>

○ <u>Colloque : date de fin (Année)</u>

○ <u>Conférence invitée (oui/non)</u>

○

○ <u>Projets ANR (multicritères)</u>

○ <u>Projet ANR : Acronyme</u>

○ <u>Projet ANR : Acronyme du programme</u>

○ <u>Projet ANR : Code décision (référence)</u>

○ <u>Projet ANR : Nom</u>

○ <u>Projet ANR : Identifiant interne</u>

○ <u>Projet ANR : État dans le référentiel</u>

○

○ <u>Projets européens (multicritères)</u>

○ <u>Projet européen : Acronyme</u>

○ <u>Projet européen : Identifiant de l'appel à projet</u>

○ <u>Projet européen : Référence</u>

○ <u>Projet européen : Nom</u>

○ <u>Projet européen : Date de fin</u>

○ <u>Projet européen : Financement</u>

○ <u>Projet européen : Date de début</u>

○ <u>Projet européen : État dans le référentiel</u>
## 2 Related work

So far, mapping has been proposed for patents mainly to represent the technologies used in certain fields or companies [3][4][5]. [1] proposes to visualize conflicts between patents, [2] proposes to identify TRIZ trends for given technologies.

Contradiction mining is well known theme in TRIZ domain but not much approaches tackled this problem as understanding the content of a patent remains a very difficult task. Despite the importance of this challenge and the NLP techniques that have been developed, very few have tackled this problem. However, we find classification of inventive principles [6][7][8], extraction of parameters [9][10], reconstruction of TRIZ matrices for targeted domains [11].

However, these methods often use simplifications such as the reduction of the number of inventive principles [6][7], the use of keywords or key phrases [8][9] or assumptions on the structure of patents [10] which makes these approaches unusable in practice on "new" content.

## 3 Contradiction mining

Understanding which contradiction a patent addresses is a very difficult task. Indeed, many patents are, in fact, not inventive (as Altshuler himself concluded) and therefore do not resolve a contradiction. Moreover, the drafting of a patent depends very much on who wrote it and the contradiction that the patent resolves, if there is one, is not always explicitly cited. All these limitations mean that the analysis process must be fine-tuned and have a validation mechanism.

### 3.1 Identification of the areas of interest

The preliminary step to identifying contradictions is to identify the areas that are likely to be most interesting. Patents have the advantage of being structured content. Some parts are indispensable, such as the abstract, the description or the claims. Other parts are not always present or are sub-parts of the description, such as the summary or the state of the art. A preliminary manual study convinced us to use the state of the art as a source of information concerning the contradictions that the patent seeks to resolve. Indeed, one often finds structures such as "To improve (EP 1), Patent ... proposes to .... However, this implies that (EP 2) is degraded". We found that the contradictions encountered by the state of the art were in fact the contradictions that the patents in question sought to overcome. The textual analysis will therefore only be carried out on the part of the state of the art of the patents which is previously extracted.
○ Projet européen : Identifiant interne

○

○ Date de production : année

○ Date de mise en ligne : année

○ Date de publication : année

○ Date d'écriture : année

○ Date de modification du dépôt : année

○ Date de dépôt : année

○ Date de publication électronique : année

○

○ Collection HAL (multicritères)

○ Collection HAL : catégorie

○ Collection HAL : Code

○ Collection HAL : Nom

○ Collection HAL : Identifiant interne

○

○ Identifiant interne du contributeur/déposant

○ Nom complet du contributeur/déposant

○

○ Domaines

○ Domaine primaire

○ Domaine racine

○ Sous-domaine niveau 1

○ Sous-domaine niveau 2

○ Sous-domaine niveau 3

○

○ Statut du document

○ Version du document

○ Type de dépôt

○ Type de document

○ ISBN

○ Numéro - référence

○ Identifiant DOI

○

○ Classification

○ Audience

○ Vulgarisation

○ Comité de lecture - texte (oui ou non)

○ Actes de colloque

○ Référence interne
## 3.2 Mining process

The extraction process is separated into three distinct phases. The first phase is an automatic summary phase with the selection of sentences with the highest probability of containing one or more parameters related to the contradiction that the patent allows to solve. For example, in the US5316377 patent two distinct sentences can be selected:

"For limited use or lightweight applications, such as with barbecue carts, lawn mowers, trash containers and many other devices, plastic wheels can serve the same purpose, but at relatively lower costs."

"Inherent negatives of such wheels, however, are that the core of the wheels are hollow and thus the wheels tend to be noisy."

The first of these two sentences isolates a "lightweight" evaluation parameter, while the second isolates "noisy". We are therefore dealing here with a weight / noise contradiction. The first of these sentences will be called the "first part of the contradiction" while the second will be called the "second part of the contradiction". Automatic summarization is a common task in language processing. Deep neural networks such as transformers [15] are used. These neural networks have become popular in recent years in all language processing tasks thanks to their ability to be pre-trained. The best known of these is BERT (Figure 1).

<figure><img src="image_2.png" /><figcaption>Figure 1: BERT[12], a pretrained deep neural network</figcaption></figure>

BERT[12] was introduced by Google in 2019 and allows to generate contextual representations of tokens for textual content. These vector representations can then be used to classify tokens, documents, etc. These tools have replaced non-contextual representations such as Word2Vec[16] which are less efficient. Indeed, with contextual representations, the same word can have different representations depending on the words that are close to it in the sentence or paragraph. This makes it possible to integrate better quality information on the meaning of words. It is relatively easy to use this type of network for automatic summarisation (Figure 2). An additional
○ Financement

○ Collaborations

Ajouter

Lancer la recherche

Recherche experte (SolR)

Recherche experte (SolR)

Lancer la recherche
Vers la recherche avancée

• Déposer

X

X

X

Loading...

X

Le document que vous demandez est en access restreint
jusqu'au 2025-01-01 !

Revenir a la notice

Retourner à l'accueil

[image]

[image]

[image]

[image]

Contact

• Support

Ressources

• Documentation

• FAQ

• API
Transformer layer on top of the encoder is used to combine the special CLS token representations intended to be the sentence representations [14]. The decision is then made by two (binary) sentence classifiers which will select the best first part of the contradiction and the best second part of the contradiction.

<figure><img src="image_3.png" /><figcaption>Figure 2: BERT for extractive summarization</figcaption></figure>

A document classifier is associated to the summarization model to validate the ex-
traction. It is supposed to predict whether there is a contradiction to mine. If it pre-
dicts that the patent indeed contains a contradiction the sentences considered as first
and second part of contradictions are assumed to be correct and the process continues.
If not, the results are ignored.

The last step of the process is the extraction of parameters. As this step is very sim-
ilar to a Named Entity Recognition (NER) task, the model used will be a classical
NER Encoder + Classifier model. Named Entity Recognition is a token-wise classifi-
cation. In general, it consists of finding tokens defining the names of people or loca-
tions but in our case we will introduce three classes with Evaluation Parameters (EP),
Action Parameters (AP) and a rejection class. An XLNet[13] type encoder is used as
it is one of the most efficient currently. It is also a pre-trained encoder like BERT. A
Conditional Random Field is added on top of the encoder to improve consistency in
• OAI-PMH

• AuréHAL

## Informations

• À propos

• Données personnelles

• Mentions légales

• Accessibilité

• Conformité RGAA

## Questions juridiques

• Je publie, quels sont mes droits ?

• Loi pour une République numérique

• Stratégie de non-cession des droits

• SHERPA/RoMEO

## Portails

• Portails institutionnels

• HAL

• HAL SHS

• HAL Thèses

• MédiHAL

## CCSD

• CCSD

• Episciences

• Sciencesconf
<figure><img src="image_4.png" /><figcaption>Figure 3: Parameters mining model</figcaption></figure>

predictions. Indeed, with a linear classifier for example, the predictions for each token are independent of each other, i.e. if a token is thought to belong to an EP, the following token will not be influenced by this prediction. This is actually the case as PEs are often formed by several words, so when a token is labelled as a potential PE, the following tokens are more likely to be part of it too. The model used is shown in Figure 3. We therefore have a complete process allowing us to extract from a patent the contradiction(s) that it seeks to resolve. This will allow us to find the main research directions in the field from a set of patents. However, if keeping the original parameters of the patents (thus applied to the domain) also makes it possible to represent the domain correctly, we will see that in the optics of keeping a constant form of representation we fall back on the generalized TRIZ contradictions with the 39 original parameters.

# 4 Mapping construction

From a multi-criteria search, our mapping tool allows to select a subset of patents that will be used to map the domain. After running the mining process, a selection of best phrases appears for each patent. The left part gathers the best candidates for the first part of the contradiction while the left part gathers the best candidates for the second part of the contradiction (Figure 4).

The parameters of the first sentence of each part are then translated into the original TRIZ parameters before being displayed. This translation is done by similarity. An embedding is constructed for each parameter and this is then compared by cosine similarity with those of the TRIZ parameters. Experimentally, we have shown that building the embedding from a sentence Parameter + must be studied allows to build
<figure><img src="image_5.png" /><figcaption>Figure 4: Output of our mining process</figcaption></figure>

better embeddings than by taking only the few tokens contained in the parameters. Indeed, the model used for the construction of the embeddings is a model learned on sentence similarity datasets and it is therefore logical that it works best with sentences.

## 5 Visualization

<figure><img src="image_6.png" /><figcaption>Figure 5: Mapping for 100 molding patents</figcaption></figure>
The map (Fig. 5) has the form of a 39*39 matrix. The first part of the contradiction is on the ordinate while the second part of the contradiction is on the abscissa. We only deal here with contradictions between two parameters. In the quite frequent case where several evaluation parameters are in contradiction with several evaluation pa- rameters (if there are several evaluation parameters in one or more of the selected sentences in the patent). In this case, the contradictions are decoupled and it is consid- ered that each of the parameters in the first part of the contradiction will be in contra- diction with each of the parameters in the second part of the contradiction. For exam- ple, in Figure 4, reliability is considered to be in contradiction with "laborious" and "cost intensive process", and the same is true for "safety". The color scale is then adjusted with a maximum intensity for the most common contradiction. In this exam- ple, patents with "molding" in the title were used. It can be seen that the most com- mon contradiction is between parameters 29 and 39 i.e. manufacturing precision and productivity. This would of course require each patent to be checked by an expert in the field but at first sight the results do not seem to be outliers.

# 6 Conclusion and perspectives

In this paper we have presented an algorithm to build a multi-domain or single-domain mapping of contradictions. This allows to have, in a few minutes, a visibility on a domain that would otherwise require hours of reading by engineers. The contra-diction mining process is separated into three phases with a first phase of automatic summarization to isolate the sentences containing the parameters of the contradiction resolved by a patent, the validation of the extraction and finally the extraction of the parameters from the selected sentences. The parameters are then translated into TRIZ parameters to allow multi-domain mapping if required.

Future work includes the exploitation of this pipeline to search for correspondences between a given problem and solutions. In this way, a real solution search engine can be set up.

# References

1. Zheng Li, Mark Atherton, David Harrison: Identifying patent conflicts: TRIZ-Led Patent Mapping. In World Patent Information, Volume 39, 2014, Pages 11-23

2. Janghyeok Yoon, Kwangsoo Kim: An automated method for identifying TRIZ evolution trends from patents, Expert Systems with Applications, Volume 38, Issue 12, 2011, Pages 15540-15548

3. Russo, David et al. "A new patent-based approach for technology mapping in the pharma- ceutical domain." Pharmaceutical patent analyst (2013): 611-27.

4. Guo, Xin & Park, Hyunseok & Magee, Christopher. (2016). Decomposition and Analysis of Technological domains for better understanding of Technological Structure.

5. Arho Suominen, Hannes Toivanen, Marko Seppänen: Firms' knowledge profiles: Mapping patent data with unsupervised learning, Technological Forecasting and Social Change, Volume 115, 2017, Pages 131-142
6. Han Loh, Cong He, and Lixiang Shen. "Automatic classification of patent documents for TRIZ users". In: World Patent Information 28 (Mar. 2006), pp. 6–13. doi: 10.1016/j.wpi.2005.07.007.

7. Cong He and Han Tong Loh. "Grouping of TRIZ Inventive Principles to facilitate automatic patent classification". In: Expert Syst. Appl. 34 (2008), pp. 788–795

8. Y. Liang et al. "Computer-aided classification of patents oriented to TRIZ". In: 2009 IEEE International Conference on Industrial Engineering and Engineering Management. 2009, pp. 2389–2393.

9. Hsiang-Tang Chang, Chen-Yen Chang, and Wen-Kuei Wu. "Computerized innovation inspired by existing patents". In: May 2017, pp. 1134–1137.

10. Gaetano Cascini and Davide Russo. "Computer-aided analysis of patents and search for TRIZ contradictions". In: International Journal of Product Development 4 (Jan. 2007), pp. 52–67

11. Daria Berdyugina and Denis Cavallucci. "Setting Up Context-Sensitive Real-Time Contradiction Matrix of a Given Field Using Unstructured Texts of Patent Contents and Natural Language Processing". In: Systematic Complex Problem Solving in the Age of Digitalization and Open Innovation. Ed. by Denis Cavallucci, Stelian Brad, and Pavel Livotov. Cham: Springer International Publishing, 2020, pp. 30–39

12. Jacob Devlin et al. "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding". In: NAACL-HLT. 2019.

13. Zhilin Yang et al. "XLNet: Generalized Autoregressive Pretraining for Language Understanding". In: Advances in Neural Information Processing Systems 32. Ed. by H. Wallach et al. Curran Associates, Inc., 2019, pp. 5754–5764.

14. Yang Liu and Mirella Lapata. "Text Summarization with Pretrained Encoders". In: EMNLP/IJCNLP. 2019.

15. Ashish Vaswani et al. "Attention is All you Need". In: Advances in Neural Information Processing Systems 30. Ed. by I. Guyon et al. Curran Associates, Inc., 2017, pp. 5998–6008

16. Mikolov, Tomas & Chen, Kai & Corrado, G.s & Dean, Jeffrey. (2013). Efficient Estimation of Word Representations in Vector Space. Proceedings of Workshop at ICLR. 2013.