# Bringing Together Engineering Problems and Basic Science Knowledge, One Step Closer to Systematic Invention

Omar Boufeloussen, Denis Cavallucci

<figure><img src="image_1.png" /></figure>

## To cite this version:

Omar Boufeloussen, Denis Cavallucci. Bringing Together Engineering Problems and Basic Science Knowledge, One Step Closer to Systematic Invention. TRIZ Future conference, Sep 2021, Bolzano, Italy. pp.340-351, 10.1007/978-3-030-86614-3_27. hal-04067822

HAL Id: hal-04067822

https://inria.hal.science/hal-04067822

Submitted on 13 Apr 2023

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_3.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# Bringing Together Engineering Problems and Basic Science Knowledge, One Step Closer to Systematic Invention

Omar Boufeloussen¹ and Denis Cavallucci¹

¹ INSA Strasbourg, ICube Lab Strasbourg, France
24, Boulevard de la Victoire, 67084 Strasbourg Cedex, France
omar.boufeloussen@insa-strasbourg.fr, denis.cavalluccinsa-strasbourg.fr

**Abstract.** Since its origins, TRIZ theory has been concerned with the use of fundamental knowledge of physics as a means of solving engineering problems. The three decades of TRIZ history have seen the emergence of methodological tools such as substance-field analysis combined with databases that have become increasingly computerized in line with advances in computer science. However, the current revival of artificial intelligence calls into question everything that has been done previously in terms of classification and allows us to think about the pairing of engineering problems and knowledge of physics not from closed databases, but in real time from online data sources and according to the versatility of web content. This article presents a new approach to pairing called PhysiSolve based on Artificial Intelligence techniques. We used natural language processing models like transformers based on attention to boost learning which allows us to outperform classical models for downstream tasks and unlock technical language understanding to automate data classification and facilitate semantic search for better ideas generation. Our research has led us to develop an online tool whose first results are presented and discussed from a perspective of measuring the efficiency of conducting an inventive activity. These results reinforce our belief that artificial assistance to inventiveness in R&D is no longer just possible but paves the way for a new era of digital tools for engineers and industrial companies.

**Keywords:** TRIZ, Deep Learning, Machine Learning, Natural Language Processing, Artificial Intelligence.

## 1 Introduction

TRIZ [1] is a pragmatic approach to invention that is supported by years of experience and work of an entire community, headed by Altshuller, assisted by handful enthusiasts, and then by thousands of equally passionate users today throughout the world.

Like all approaches born on the enthusiasm of their creators, a legitimate expertise is built up over decades and the informed reader knows the years of learning that are necessary to practice TRIZ efficiently. But like all approaches with a growing number
of satisfied users, an equally large number of detractors criticize TRIZ and thus stimulate (one should always listen to them) the research that pushes the theory into the future. These criticisms can be summarized in three categories. The first is related to the complexity of learning. TRIZ is perceived as requiring years of practice and mentoring to be deployed effectively. This is time that is no longer available to companies, most of whom expect a tool to be quick to learn and simple to use. The second is linked to the original lack of formalism of the TRIZ. It leads to the observation that the user is sometimes exposed to having to interpret what the approach expects of him or her, and this appears to be more of an art than a science. Finally, the third category of criticisms is linked to the dependence of the method on the user's experience. If a person has little technical hindsight and little scientific culture, his ideas sometimes lack originality because the connections are not made between what the approach leads to interpret (a principle, a standard, a law) and the user's know-how. However, the exhaustiveness of the approach depends on maximizing the chances that it generates a bridge between a problem and its solution.

Consequently, a direction, essential for TRIZ, is that of putting it into an equation to automate some of its intrinsic mechanisms. But knowing that they are part of human creativity, it is indeed a form of axiomatization of creative and inventive mechanisms (underlying TRIZ) that we are talking about here. A task that has preoccupied artificial intelligence scientists for decades. The first question that comes to mind is: Is this even possible? Isn't the creativity of human brain the last bastion untouched by AI? Our goal is not that far-fetched. But it is nevertheless very ambitious, since it aims at automating the TRIZian inventive process by artificial intelligence. It is the immense dynamism of the AI community that makes this goal potentially achievable as its progress, often open to communities, allows researchers to move on.

In this article we report on an artificial approach to assist the inventive process inspired by TRIZ based on the use of the contradiction model and on the automatic association between oppositions between parameters and elements of the free literature present on the web. Our experiments are oriented towards Wikipedia and a set of target sites where serious, innovative and regularly updated information resides. Our intentions are to make sure that any problem finds instantly, in a sufficiently large multidisciplinary base, the most eligible piece of information to solve its problem. The question here is therefore: "Can the maturity of AI algorithms and techniques allow for the production of a TRIZ-like intellectual mechanism? "In essence, the reader will have understood that if this first step is taken, it will open the door to the (TRIZ) challenge of human versus machine.

After this introduction, the article proposes a review of the literature. Then, in a Methodology section, we present our approach, the tools and techniques that we have assembled and particularized to the TRIZ. Then comes the exercise of testing our tool on a case, observing the results obtained and analyzing their scope compared to their interest in the resolution of an inventive problem. Then a discussion section and a conclusion finalize this article.
## 2 Literature review

Although AI-based research in TRIZ is not so numerous, we still find in literature existing teams having performed research in the orientation. In this part we review several researches relevant from our challenge and that constituted literal investigations for our researches.

### 2.1 AI-based TRIZ orientations

The inventive design process is not intuitive for non-users of Triz and not practical as it's not easy to understand and use inventive principles if you are not familiar with them due to the abstraction of their definition and semantic similarities between some inventive principles which create some confusion to Triz users[2]. For instance, according to [3] the 40 inventive principles are divided into 33 distinct inventive principles, they were easily and explicitly identified from text information and the other 7 inventive principles are hardly recognized because they have no specific text information to identify them, those principles are called obscure principles.

Automatic inventive design requires automatic classification of inventive principles, so to classify patents[3] into inventive principles, it takes classification models to distinguish between inventive principles textually and semantically. For example, according to [3] by collecting patents, they found there's some inventive principles that share the same textual and semantics information for example both principles self-service and turn the harm to one's good share the same meaning.

Tate & all [4] have used also natural language processing and machine learning to explore patents data to evaluate patents ideality and level of invention. As patents and scientific papers are the source of inventive ideas, Authors [5] have made use of advanced text analysis in order to extract semantic concepts to facilitate searching and navigating through patents and scientific publications. According to [6] text mining techniques and statistical approach based on word frequencies level are implemented to classify patents into inventive principles. Manual extraction is one of the methods to find inventive solutions in patents, [7] have trained an embedding model which is Doc2Vec to learn semantic relations for technical contradictions extraction, this method is effective as way to automate information extraction, but the model is limited at capturing contextual information. In the context of multidisiplinarity, nature is one of the sources of inventive ideas to solve technical problems, S. Palak and P. Manojkumar [8] have explored this idea by linking and classifying real world problems to nature inspired algorithms to get solutions inspired from nature. This approach has brought a method for selecting the best candidate for solving a specific problem but does not rely on contradiction formalism to reach a TRIZ-like formulation. Thus, the studies that have been done before struggles with three main issues. First, the insufficient quantity of labeled patents with inventive principles because all classical machine learning models takes a lot of data for training to learn properly [9]. Secondly, they do not focus on classifying patents into all 40 inventive principles [10], for example, they've limited their study on only classifying patents into 6 inventive principles [3]. The same thing is done in the other study by grouping the most similar inventive principles in one group.
Finally, one cannot write about physical effects without thinking about Altshuller's way of classifying effects in Functions (pointers) and various tools either free and online (like Oxford Creativity or Creax) or from a software provider like IHS and Goldfire. These solutions are in a different category, we are talking about a human-indexed database, while what we propose is an automated way of storing unstructured web content real-time. As a perspective of such an approach, if new data were arriving, there won't need any human involvement for new data's to be classified, but in our branch named physics (see Fig. 1), this action is fully automated.

# 3 Methodology

This section is about presenting the process behind the design of the three functionalities of our model. Since we decided to install it on a server online and cooperate with industrial users in order to test the validity of our approach, we named it PhysiSolve.

## 3.1 First, collecting data

PhysiSolve contains three major functionalities. They associate user's problem to information from various filed of knowledge (Physics, Engineering, Biology) Fig. 1.

<figure><img src="image_4.png" /><figcaption>Fig. 1. Flowchart of PhysiSolve functionalities</figcaption></figure>
The first database contains more than 1000 physical effects to answer straight for-
warded textual requests usually formulated by designers as action verb followed by a
complement either completed by some parameters, field, or state of matter. To find the
most relevant physical effects and applications for a given use case, we used Wikipedia
articles as a source to retrieve a large quantity of physical effects and applications. To
do so we developed a python script to scrap and filter the metadata of those physics
effects as show in Fig. 2.

<figure><img src="image_5.png" /><figcaption>**Fig. 2.** Physics effects scraping process</figcaption></figure>

By applying a similar approach, engineering and biology articles are scraped in order
to allow users to formulate their problems as contradictions and browse in return the
most relevant engineering and biological articles matching an inventive principle so that
they can come up with a practical solution to their problems.

**3.2 Using and adjusting existing AI-based tools**

In order to automate inventive design by recommending contents from the three func-
tions of PhysiSolve, we developed natural language models specific to each function.
The purpose of the first function is to return the most relevant physical effects to the
user's requests, so to achieve this we implemented a method of semantic search based
on transformers architecture Fig. 3.

<figure><img src="image_6.png" /><figcaption>Fig. 3. The transformer architecture</figcaption></figure>
The transformer architecture is a box of connection of two components, encoder component and the decoding component [11]. The encoder component represents a stack of encoders and in the same manner decoder component is a serie of decoders Fig. 4. This architecture is designed for machine translation and what makes it powerful compared to other models is adding self-attention blocks which is a layer that helps the encoder to look at other words in the input sentence as it encodes a specific word [12]. Actually, when the model processes a word, the self-attention block allows learning the weights of each word in the sentence in order to get more information about the word processed by taking into account the words close to it.

<figure><img src="image_7.png" /><figcaption>Fig. 4. Encoder and decoder blocks</figcaption></figure>

Based on this architecture, the model BERT [15], is used to build a Siamese Network [14] which allows learning a powerful representation of the semantics of the text due to training BERT on supervised tasks like words in a sentence from the surrounding context using a large corpus like Wikipedia articles. Siamese network is combination of two parallel similar neural networks, the first one produces the vector of a sentence one and the second one produces the vector of a sentence two as shown in Fig. 5.
<figure><img src="image_8.png" /><figcaption>**Fig. 5.** Architecture of siamese network</figcaption></figure>

To train siamese network, cosine similarity is used to calculate the distance between the two vectors u, v in Fig. 5 then the mean squared-error loss as the objective function so we can minimize the distance between two sentences with the same meaning and maximize it when they are dissimilar. For our purpose we used the pre-trained siamese network called sentences transformers to compare the query against all physics effects and return the most semantically close effects based on cosine similarity score as presented below Fig. 6.

<figure><img src="image_9.png" /><figcaption>**Fig. 6.** Semantic search for physics effects</figcaption></figure>

Moreover, we have worked also on fine-tuning BERT for a binary classification model Fig.7 to distinguish between articles about physical effects and other articles so we can extend our database of effects by scraping online articles and filter only physical effects.
<figure><img src="image_10.png" /><figcaption>**Fig. 7.** Finely tuned BERT for binary classification</figcaption></figure>

By scraping articles from different domains and in combination with physics effects we trained our binary classifier on 1000 physical effects and 1000 articles from different domains. Results were evaluated compared to 15 TRIZ user cases found in internet and 13 attempts have brought identical automated results. To evaluate the performance of the model we use accuracy as a metric because we have a balanced dataset, it reached almost 98% on test data and as it did on validation without overfitting Fig. 8. Thus, we can proceed with the model for production to sort out physical effects.

<figure><img src="image_11.png" /><figcaption>**Fig. 8.** Accuracy and loss function curves</figcaption></figure>

When it comes to the second part of PhysiSolve, we have 2600 engineering news labeled with inventive principles but it is obviously not enough to build a robust multi-label classifier for 40 inventive principles. Thus, we decided to implement the same logic of semantic search by comparing news articles and 40 inventive principles and set a high score in order to affect to news articles the closest inventive principles. This way, the user formulates its query by building the problem model. Then by automatically labeling news articles, we can not just return inventive principles but also engineering news demonstrating the applications of those inventive principles.

<figure><img src="image_12.png" /><figcaption>**Fig. 9.** Semantic search for engineering and biology articles</figcaption></figure>
# 4 Case Study

In this section, we present an example on how some functions of Physisolve work. One of the most important steps to test our models is using an API to deploy siamese model and make use of it by producing the right output for the user and communicate the results through linking the front ends with APIs Fig.10.

<figure><img src="image_13.png" /><figcaption>**Fig. 10.** Deployment of DL models</figcaption></figure>

## 4.1 Automating Engineering articles association

The first step is to formulate the contradiction problem according to IDM-TRIZ [13] as this approach is more accurately defined in terms of contradiction axiomatization. Once the problem is defined and the inventive principle is selected based on Altshuller's statistics in his Matrix we use semantic search API Fig. 11 for engineering articles to return all articles that have been matched automatically with the right inventive principle. In this example we took an article appearing on MachineDesign April 30th 2021 entitled: "Lattice Design Enables 3D-Printed Nasal Swab Production". Our approach sent back IP30th for interpreting this article content.
<figure><img src="image_14.png" /><figcaption>**Fig. 11.** Example of semantic comparison between MachineDesign's article and inventive principles</figcaption></figure>

In this example, the article is matched with all inventive principles that are sorted according to their cosine similarity score.

## 4.2 Automating physical effects association

Concerning physical effects, the way the problem is formulated is intuitive using natural language. It is just a textual query defined by a user in order to search for solutions in the effects database. It helps instead of being forced to search in a restricted classification problems statements (like the old way) to find the corresponding physics effects, the user can formulate its query freely and without any specific criteria as shown in the example Fig. 12. For this test we typed "How to waterproof a rooftop" and our API sent back superhydrophilicity on top of the results.

<figure><img src="image_15.png" /><figcaption>**Fig. 12.** Example of semantic comparison of user's query and physics effect database</figcaption></figure>
# 5 Discussion

In the previous exercise, we have to our acknowledge in TRIZ publications, used Artificiel Intelligence tools to pair a problem formulated using natural language (how to warterproof a rooftop?) and unstructured knowledge scraped from the web. Our methodology brings us to superhydrophilicity which is a physical effect known for its particular properties and which consists in repelling water. Such an effect could of course have been found in the mind of a person who already knew about this effect. But in our approach, no word similar to the question appears in the texts that deal with this subject. Thus, by classical means of internet research, this result is likely to be drowned in a multitude of pages that deal with more or less similar subjects. Add to that the fact that this result took 0.1 seconds to arrive on the screen, and are then classified with other results of physical effects to explore and whose proximity score is encouraging (higher than 0.1). Second test, we use a randomly taken news in MachineDesign and ran our API. The result shows IP30 as an automated association from Lattice structure expression. This first attempt to mimic the inventor's reasoning by supervising unstructured text associations and sentences written in natural language offers promising results and opens the way to automating TRIZian tasks in software to assist a form of inventive R&D.

However, our approach has its limits, it has not yet reproduced the multidisciplinary synthesis that Altshuller was able to produce by synthesizing patents from all the disciplines present in the corpus of world patents. Consequently, we need to perform a much larger study on a larger quantity of data and number of experiments to be able to draw a more reliable conclusion. Finally, the approach of starting from a problem and extracting a contradiction is the subject of other research of our team and that of mapping knowledge from patents as well. Associating problem models of a given discipline with the expert language of this domain to better instantiate a resolution approach is also part of our team's work. The research subject of this article involves the "last link" of a long chain of automation of inventive reasoning and is, only now, proposing its first convincing results.

# 6 Conclusion

Our world is in its digital age, and all departments of the company are affected by the Industry 4.0 paradigm. Until now, R&D has been little affected by this digitization because inventive reasoning was only slightly challenged by digital technology. But the advent of powerful artificial intelligence techniques such as Machine Learning or Deep Learning and Natural Language Processing is now progressing exponentially. Ten years ago, we were producing slow and not very exhaustive methodological results through hours of exchanges between experts who were not very available. With the proposed approach, the speed that separates the posing of a problem and the proposed intelligent answer (beyond a simple internet request) is now only a few milliseconds. It sometimes surprises because it provides answers that are outside the user's field of competence. At first glance, such an approach could arouse mistrust, but it is in the ordering
of answers and in the art of asking the right questions that our next research projects will be situated. For the time being, they also suffer from the lack of massive data that is free to access and use. The paradigm of open data and datalake, here again, suggests that in the coming months, it will certainly be possible to considerably evolve the value of the results produced according to the mass of data available to be investigated. Perhaps then, the challenge between man and machine will produce results such that the company will no longer be able to do without such tools.

## References

1. Terninko, J., Zusman, A., Zlotin, B.: Systematic Innovation: An Introduction to TRIZ (Theory of Inventive Problem Solving). (1998).

2. Cong, H., Tong, L.H.: Similarity between TRIZ Principles. Triz J.

3. Cong, H., Tong, L.H.: Grouping of TRIZ Inventive Principles to facilitate automatic patent classification. Expert Syst. Appl. 34, 788-795 (2008).
https://doi.org/10.1016/j.eswa.2006.10.015.

4. Adams, C., Tate, D.: Computer-Aided TRIZ Ideality and Level of Invention Estimation Using Natural Language Processing and Machine Learning. Growth Dev. Comput.-Aided Innov. 27-37 (2009).
https://doi.org/10.1007/978-3-642-03346-9_4.

5. Kaliteevskii, V., Deder, A., Peric, N., Chechurin, L.: Conceptual Semantic Analysis of Patents and Scientific Publications Based on TRIZ Tools. Syst. Complex Probl. Solving Age Digit. Open Innov. 54-63.
https://doi.org/10.1007/978-3-030-61295-5_5.

6. Liang, Y., Tan, R., Wang, C., Li, Z.: Computer-aided classification of patents oriented to TRIZ. IEEE Int. Conf. Ind. Eng. Manag. (2009).
https://doi.org/10.1109/IEEM.2009.5372983.

7. Zhai, D., Li, M., Cai, W.: TRIZ Technical Contradiction Extraction Method Based on Patent Semantic Space Mapping. 11th Int. Conf. E-Bus. Manag. Econ. 125-130 (2020).
https://doi.org/10.1145/3414752.3414802.

8. Sukharamwala, P., Parmar, M.: Mapping of Real World Problems to Nature Inspired Algorithm using Goal based Classification and TRIZ. Procedia Comput. Sci. 171, 729-736 (2020).
https://doi.org/10.1016/j.procs.2020.04.079.

9. Géron, A.: Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow. O'Reilly Media, Inc.

10. Tong, L.H., Cong, H., Lixiang, S.: Automatic classification of patent documents for TRIZ users. World Pat. Inf. 28, 6-13 (2006).
https://doi.org/10.1016/j.wpi.2005.07.007.

11. Alammar, J.: The Illustrated Transformer, https://jalammar.github.io/illustrated-transformer/, last accessed 2021/04/02.

12. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Po-losukhin, I.: Attention is all you need. NIPS17 Proc. 31st Int. Conf. Neural Inf. Process. Syst. 6000-6010, (2017).

13. Liu, C.-C., Chen, J.L.: A TRIZ Inventive Design Method without Contradiction Information. (2001).

14. Reimers, N., Gurevych, I.: Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. Proc. 2019 Conf. Empir. Methods Nat. Lang. Process. 3982-3992 (2019).

15. Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. 13 (2018).