# PatentInspector: An Open-Source Tool for Applied Patent Analysis and Information Extraction
## 4 Architecture and Workflow
### Data Collection, Preprocessing and Storage  
PatentInspector operates on patent record data offered free of charge by the USPTO. More specifically, the database used in PatentInspector relies on bulk data available in the PatentsView platform of USPTO [110], which serves as a repository of all USPTO-registered and granted patents and is updated regularly. The tool includes a management utility named \"USPTO\", showcased in Figure 3, that automates the process of downloading, decompressing, preprocessing and inserting the data into the database. This repository is organized in tables, with each table containing a different aspect of patent records (e.g., patent classes, patent inventors, etc.) PatentInspector emphasizes only targeted tables made available from USPTO and, more specifically, only those regarding granted patents, while it does not retrieve those who have applied to USPTO but have not yet received a patent grant. This was a conscious decision based on the rationale that applied patents may be rejected by the patent office and would thus hold reduced importance in the collected data [111]. The different fields and tables retrieved are presented in Table 2. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Table Name** & **Description** & **Fields** \\ \hline g\_location\_disambiguated & Locations of inventors and assignees & location\_id, country, state, city, longitude, latitude, county\_fips, state\_fips \\ \hline g\_cpc\_title & CPC schema & cpc\_subclass, cpc\_subclass\_title, cpc\_group, cpc\_group\_title, cpc\_class, cpc\_class\_title \\ \hline g\_patent & Patent information for granted patents & patent\_id, patent\_type, patent\_date, patent\_title, patent\_abstract, num\_claims, withdrawn \\ \hline g\_application & Application information of granted patents & patent\_id, filing\_date \\ \hline g\_figures & Number of drawings and drawing sheets & patent\_id, num\_figures, num\_sheets \\ \hline g\_cpc\_current & CPC classification of granted patents & patent\_id, cpc\_group \\ \hline g\_pct\_data & PCT data of granted patents & patent\_id, published\_or\_filed\_date, filed\_country, pct\_doc\_number, pct\_doc\_type \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Fields retrieved from the USPTO. Figure 3: An overview of the USPTO management utility. After the tables of interest, containing information about roughly eight million patents, are downloaded, an automated preprocessing procedure takes place. The preprocessing deployed in PatentInspector involves stop word removal and the lemmatization of text fields such as the patent's title or abstract, to facilitate and accelerate the text analysis performed in later stages. Additionally, in this phase, computations are performed in advance for optimization purposes and stored as additional columns effectively, constructing a sort of long-term database stored cache. Table 3 summarizes all the precomputed fields that aid the throughput of the application. After the preprocessing is finished, the entire set of patent records is inserted into the database. This is achieved through two different approaches depending on the size of the data. For small tables such as Location, the Django ORM is leveraged to insert the data. For larger tables such as Patent, the preprocessed chunks are stored in a bulk CSV in the file system, which is later loaded into the database using Postgres' COPY command, resulting in a significant performance boost. The schema of the database for computational-related and user-related tables is shown in Figures 4 and 5, respectively.