# MEXN: Multi-Stage Extraction Network for Patent Document Classification
## 1 Introduction  
Recently, neural network methods have various influences on the natural language process, including automatic document classification problems. A typical example of automatic document classification used in real life is the automatic patent classification (APC), which is a useful way to mitigate the enormous cost of analyzing patent documents as the number of patent documents increases. With the dramatic rise in the adoption of deep learning in APC in the past few years, recent studies [1; 2; 3; 4; 5] tried to learn the characteristics of the patent documents (e.g., novelty and inventive step) from the keywords itself the patent documents itself. In the early approaches, a simple convolution neural network (CNN) has been used to learn the semantic meaning of the documents [1]. Then, integrated model approaches [4; 5; 6] have been introduced for a deeper understanding of the semantic meaning of the documents, which has frequently combined CNN with recurrent neural networks (RNN [7]). Jointly training their combined architecture discriminatively allows one to leverage the expressive power of deep neural networks. Furthermore, a self-attention mechanism is also adopted to capture long-range interactions of the patent documents [3]. These methods show that the general document classification model can be applied to the patent document classification work even if the entire contents of the patent are not used.  
Despite the success of recent works, APC is still challenging as they have designed deep network architectures without deeply understanding the contents of novelty and inventive step, which are typical characteristics of the patent documents. These two attributes are determined by a series of claims and descriptions, and each paragraph contains the contents of each paragraph. The combination and relationship of the patent contents plays a essential role in generating the semantic meaning of noveltyand inventive step. Therefore, in order to thoroughly analyze a patent document, it is required to design the network architecture in consideration of analyzing the \"whole\" document. Moreover, recent studies have also failed to show various metrics for evaluating multi-label classification performance, which is one of the core characteristics of patent documents.  
In this paper, we aim to better understand the patent document by hierarchically analyzing the structure of the \"whole\" patent document with marginally increasing the computational cost, taking into account the novelty and inventive step both, thereby obtaining well-balanced documents features that embrace all paragraphs (patent claims and their descriptions). We design the whole process in three stages. First, we divide the input document to fixed-length paragraphs and extract features with weight sharing networks. In the second stage, the paragraph features are summarized into a document feature by using the attention mechanism. Finally, we compute the weight matrix with hierarchical category information of the patent.  
**Contributions.** Our main contributions are summarized as follows:  
* MEXN is a novel document classification network for patent documents that analyzes structural configuration and entire contents through a multi-stage feature aggregation.
* We introduce a parallel and hierarchical network design to improve classification performance and reduce the computational cost.
* Investigations on the extendability of MEXN are conducted by proposing a structural derivative. MEXN has a flexible structure in which various existing methods can be used on each stage.
* We analyze and evaluate MEXN through extensive experiments on multiple benchmarks. The experiments show that our network achieves the great improvement of the patent document classification task.