# Using Machine Learning Language Models to Generate Innovation Knowledge Graphs for Patent Mining
## 3 Methodology
### Preprocess of Patent Text and Paragraphs  
In this study, we mainly used Google Patent to obtain the patent text. First, the patent text was obtained from Google Patent web pages as a HTML file. In this study, we analyzed the description section of the patent text, instead of the abstract section, to analyze the patent content in more detail. Specifically, we extracted all the class attributes of the description section from the HTML file and removed the HTML syntax to obtain the description section of the patent text. Then, the text is broken into smaller units by breaking up the sentences and words, and the words are word-reduced so that the words with the same meaning can be counted as the same element by the computer. The stagnant words that are less relevant to the text are further removed by the system.  
In this study, ChemDataExtractor was used for the sentence and word splitting [12]. After saving each paragraph of the text into an array, the specific desired paragraph was extracted for the sentence and word splitting. The purpose of the sentence splitting is to mark the end of a sentence with a punctuation mark at the end of the paragraph, and  
Figure 2: The system architecture shows the parallel modules of chemical process and patent summary graphs.  
then retrieve the sentences one-by-one. In the case of broken sentences, the words are taken out as separate units, according to the spaces and punctuation marks in the split sentences. In the past, the system would treat non-text punctuation marks, such as commas, as separate words in the sentence to be split. However, in the case of the division of a sentence containing a chemical noun, the nomenclature of the IUPAC is often incorrectly split due to its characteristics. Therefore, to avoid this, ChemDataExtractor uses chemical substance entity identification to avoid this situation and allow the chemical nouns to be correctly asserted.  
To prevent the computer from treating the same word as a different word in the statistical calculation due to different spellings of the word or the factors such as tense and singularity, we need to restore the word patterns by word lemmatization during the preprocessing. The first step is to convert the first letter of a word at the beginning of a sentence to lowercase, because it is capitalized in English; the second step is to obtain the main part of the word and remove the suffixes of the word. The above is the traditional word lemmatization method of text mining. This study makes some adjustments to the words belonging to chemical nouns. If there is no capital letter, then the same method is used as in the previous paragraph; if there is a capital letter, then one or more capital letters are judged to be present in the word, and the word is not restored. The reason for doing so is that most of the elements in the periodic table will be an uppercase letter (such as H for hydrogen) or an uppercase letter with a lowercase letter exist (such as Ag for silver). Therefore, when conducting the pre-processing, if the word contains only one uppercase letter, the system will read whether it exists in the periodic table; if it does not exist in the periodic table before the word restoration, or the word contains more than two capital letters, the system will presume the word is a chemical material term (e.g., NaCl) or a company name (e.g., BASF), and these words are not subject to word lemmatization.  
Lastly is the stop words removal, the main purpose of which is to remove words that are less relevant to the text to reduce the impact of these irrelevant words on the statistical results. In English, stop words usually refer to words that occur frequently but have little meaning, such as \"in\", \"on\", \"a\", \"the\", etc. The high frequency of stagnant words often leads to the misclassification of stagnant words as important words in the calculation of the keywords.