SlowGuess/ACL_2020_OCR huggingface dataset:
Ying Luo and Hai Zhao*
Department of Computer Science and Engineering, Shanghai Jiao Tong University Key Laboratory of Shanghai Education Commission for Intelligent Interaction and Cognitive Engineering, Shanghai Jiao Tong University, Shanghai, China MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University, Shanghai, China
kingln@sjtu.edu.cn, zhaohai@cs.sjtu.edu.cn
In this paper, we propose a novel bipartite flat-graph network (BiFlaG) for nested named entity recognition (NER), which contains two subgraph modules: a flat NER module for outermost entities and a graph module for all the entities located in inner layers. Bidirectional LSTM (BiLSTM) and graph convolutional network (GCN) are adopted to jointly learn flat entities and their inner dependencies. Different from previous models, which only consider the unidirectional delivery of information from innermost layers to outer ones (or outside-to-inside), our model effectively captures the bidirectional interaction between them. We first use the entities recognized by the flat NER module to construct an entity graph, which is fed to the next graph module. The richer representation learned from graph module carries the dependencies of inner entities and can be exploited to improve outermost entity predictions. Experimental results on three standard nested NER datasets demonstrate that our BiFlaG outperforms previous state-of-the-art models.
Named entity recognition (NER) aims to identify words or phrases that contain the names of predefined categories like location, organization or medical codes. Nested NER further deals with entities that can be nested with each other, such as the United States and third president of the United States shown in Figure 1, such phenomenon is quite common in natural language processing (NLP).
NER is commonly regarded as a sequence labeling task (Lample et al., 2016; Ma and Hovy, 2016;
Figure 1: An example of nested named entity mentions. Solid lines connect the starting and ending indices of inner nested entities.
Peters et al., 2017). These approaches only work for non-nested entities (or flat entities), but neglect nested entities. There have been efforts to deal with the nested structure. Ju et al. 2018 introduced a layered sequence labeling model to first recognize innermost entities, and then feed them into the next layer to extract outer entities. However, this model suffers from obvious error propagation. The wrong entities extracted by the previous layer will affect the performance of the next layer. Also, such layered model suffers from the sparsity of entities at high levels. For instance, in the well-known ACE2005 training dataset, there are only two entities in the sixth level. Sohrab and Miwa 2018 proposed a region-based method that enumerates all possible regions and classifies their entity types. However, this model may ignore explicit boundary information. Zheng et al. 2019 combined the layered sequence labeling model and region-based method to locate the entity boundary first, and then utilized the region classification model to predict entities. This model, however, cares less interaction among entities located in outer and inner layers.
In this paper, we propose a bipartite flat-graph
* Corresponding author. This paper was partially supported by National Key Research and Development Program of China (No. 2017YFB0304100), Key Projects of National Natural Science Foundation of China (U1836222 and 61733011), Huawei-SJTU long term AI project, Cutting-edge Machine reading comprehension and language model.
6408
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6408-6418 July 5 - 10, 2020. ©2020 Association for Computational Linguistics
network (BiFlaG) for nested NER, which models a nested structure containing arbitrary many layers into two parts: outermost entities and inner entities in all remaining layers. For example, as shown in Figure 1, the outermost entity Thomas Jefferson, third president of the United States is considered as a flat (non-nested) entity, while third president of the United States (in the second layer) and the United States (in the third layer) are taken as inner entities. The outermost entities with the maximum coverage are usually identified in the flat NER module, which commonly adopts a sequence labeling model. All the inner entities are extracted through the graph module, which iteratively propagates information between the start and end nodes of a span using graph convolutional network (GCN) (Kipf and Welling, 2017). The benefits of our model are twofold: (1) Different from layered models such as (Ju et al., 2018), which suffers from the constraints of one-way propagation of information from lower to higher layers, our model fully captures the interaction between outermost and inner layers in a bidirectional way. Entities extracted from the flat module are used to construct entity graph for the graph module. Then, new representations learned from graph module are fed back to the flat module to improve outermost entity predictions. Also, merging all the entities located in inner layers into a graph module can effectively alleviate the sparsity of entities in high levels. (2) Compared with region-based models (Sohrab and Miwa, 2018; Zheng et al., 2019), our model makes full use of the sequence information of outermost entities, which take a large proportion in the corpus.
- We introduce a novel bipartite flat-graph network named BiFlaG for nested NER, which incorporates a flat module for outermost entities and a graph module for inner entities.
- Our BiFlaG fully utilizes the sequence information of outermost entities and meanwhile bidirectionally considers the interaction between outermost and inner layers, other than unidirectional delivery of information.
- With extensive experiments on three benchmark datasets (ACE2005, GENIA, and KBP2017), our model outperforms previous state-of-the-art models under the same settings.
Our BiFlaG includes two subgraph modules, a flat NER module and a graph module to learn outermost and inner entities, respectively. Figure 2 illustrates the overview of our model. For the flat module, we adopt BiLSTM-CRF to extract flat (outermost) entities, and use them to construct the entity graph  $G^{1}$  as in Figure 2. For the graph module, we use GCN which iteratively propagates information between the start and end nodes of potential entities to learn inner entities. Finally, the learned representation from the graph module is further fed back to the flat module for better outermost predictions.
2.1 Token Representation
Given a sequence consisting of  $N$  tokens  $\{t_1,t_2,\dots ,t_N\}$ , for each token  $t_i$ , we first concatenate the word-level and character-level embedding  $t_i = [w_i;c_i]$ ,  $w_{i}$  is the pre-trained word embedding, character embedding  $c_{i}$  is learned following the work of (Xin et al., 2018). Then we use a BiLSTM to capture sequential information for each token  $x_{i} = \mathrm{BiLSTM}(t_{i})$ . We take  $x_{i}$  as the word representation and feed it to subsequent modules.
2.2 Flat NER Module
We adopt BiLSTM-CRF architecture (Lample et al., 2016; Ma and Hovy, 2016; Yang and Zhang, 2018; Luo et al., 2020) in our flat module to recognize flat entities, which consists of a bidirectional LSTM (BiLSTM) encoder and a conditional random field (CRF) decoder.
BiLSTM captures bidirectional contextual information of sequences and can effectively represent the hidden states of words in context. BiLSTM represents the sequential information at each step, the hidden state  $h$  of BiLSTM can be expressed as follows.
$$
\begin{array}{l} \overrightarrow {h _ {i}} = L S T M \left(x _ {i}, \overrightarrow {h} _ {i - 1}; \overrightarrow {\theta}\right) \\ \overleftarrow {h _ {i}} = L S T M \left(x _ {i}, \overleftarrow {h} _ {i - 1}; \overleftarrow {\theta}\right) \tag {1} \\ h _ {i} = \left[ \overrightarrow {h _ {i}}; \overleftarrow {h _ {i}} \right] \\ \end{array}
$$
where  $\overrightarrow{\theta}$  and  $\overleftarrow{\theta}$  are trainable parameters.  $\overrightarrow{h_i}$  and  $\overleftarrow{h_i}$  respectively denote the forward and backward context representations of token  $t_i$ . The output of BiLSTM  $H = \{h_1, h_2, \dots, h_N\}$  is further fed into the CRF layer.
CRF (Lafferty et al., 2001) has been widely used in state-of-the-art NER models (Lample et al.,
6409
Figure 2: The framework of our BiFlaG model.  $G^{1}$  and  $G^{2}$  are entity graph and adjacent graph created for GCN, each dashed line connects the start and end nodes for a potential entity. Solid red lines indicate inner entities recognized by the graph module.
2016; Ma and Hovy, 2016; Yang and Zhang, 2018) to help make better decisions, which considers strong label dependencies by adding transition scores between neighboring labels. Viterbi algorithm is applied to search for the label sequence with highest probability during the decoding process. For  $y = \{y_1,\dots,y_N\}$  being a sequence of predictions with length  $N$ . Its score is defined as follows.
$$
s (x, y) = \sum_ {i = 0} ^ {N - 1} T _ {y _ {i}, y _ {i + 1}} + \sum_ {i = 1} ^ {N} P _ {i, y _ {i}} \tag {2}
$$
where  $T_{y_i, y_{i+1}}$  represents the transmission score from  $y_i$  to  $y_{i+1}$ ,  $P_{i, y_i}$  is the score of the  $j^{th}$  tag of the  $i^{th}$  word from BiLSTM encoder.
CRF model defines a family of conditional probability  $p(y|x)$  over all possible tag sequences  $y$ :
$$
p (y | x) = \frac {\exp^ {s (x , y)}}{\sum_ {\tilde {y} \in y} \exp^ {s (x , \tilde {y})}} \tag {3}
$$
during training phase, we consider the maximum log probability of the correct predictions. While decoding, we search the tag sequences with maxi
mum score:
$$
y ^ {*} = \underset {\tilde {y} \in y} {\arg \max } \operatorname {s c o r e} (x, \tilde {y}) \tag {4}
$$
2.3 Graph Module
Since the original input sentences are plain texts without inherent graphical structure, we first construct graphs based on the sequential information of texts and the entity information from the flat module. Then, we apply GCN (Kipf and Welling, 2017; Qian et al., 2019) which propagates information between neighboring nodes in the graphs, to extract the inner entities.
Graph Construction. We create two types of graphs for each sentence as in Figure 2. Each graph is defined as  $G = (V, E)$ , where  $V$  is the set of nodes (words),  $E$  is the set of edges.
- Entity graph  $G^{1}$ : for all the nodes in an extracted entity extracted from the flat module, edges are added between any two nodes  $e_{ij} = (v_i, v_j)$ , where  $start \leq i < j \leq end$ , as shown in Figure 2, allowing the outermost entity information to be utilized.
- Adjacent graph  $G^2$ : for each pair of adjacent words in the sentence, we add one directed edge from the left word to the right one, allowing local contextual information to be utilized.
Bi-GCN. In order to consider both incoming and outgoing features for each node, we follow the work of (Marcheggiani and Titov, 2017; Fu et al., 2019), which uses Bi-GCN to extract graph features. Given a graph  $G = (V,E)$ , and the word representation  $X = \{x_{1},x_{2},\dots,x_{N}\}$ , the graph feature  $f\in \mathbb{R}^{N\times d_f}$  learned from Bi-GCN is expressed as follows.
$$
\overrightarrow {f _ {i}} = R e L U \left(\sum_ {e _ {i j} \in E} \left(\overrightarrow {W _ {f}} x _ {j} + \overrightarrow {b _ {f}}\right)\right)
$$
$$
\overleftarrow {f _ {i}} = R e L U \left(\sum_ {e _ {j i} \in E} \left(\overleftarrow {W _ {f}} x _ {j} + \overleftarrow {b _ {f}}\right)\right) \tag {5}
$$
$$
f _ {i} = [ \overrightarrow {f _ {i}}; \overleftarrow {f _ {i}} ]
$$
where  $W_{f} \in \mathbb{R}^{d_{x} \times d_{f}}$  and  $b_{f} \in \mathbb{R}^{d_{f}}$  are trainable parameters,  $d_{x}$  represents the dimension of word representation,  $d_{f}$  is the hidden size of GCN,  $ReLU$  is the non-linear activation function.  $e_{ij}$  represents the edge outgoing from token  $t_{i}$ , and  $e_{ji}$  represents the edge incoming to token  $t_{i}$ .
The features of the two graphs are aggregated to get impacts of both graphs
$$
f = W _ {c} \left(f ^ {1} \oplus f ^ {2}\right) + b _ {c} \tag {6}
$$
where  $W_{c}\in \mathbb{R}^{2d_{f}\times d_{f}}$  is the weight to be learned,  $b_{c}\in \mathbb{R}^{d_{f}}$  is a bias parameter.  $f^1$  and  $f^2$  are graph features of  $G^{1}$  and  $G^{2}$ , respectively.
After getting the graph representation  $F = \{f_{1}, f_{2}, \dots, f_{N}\}$  from Bi-GCN, we learn the entity score  $M \in \mathbb{R}^{N \times N \times L}$  for inner layers as
$$
M _ {i j} = \operatorname {s o f t m a x} \left(W _ {3} \operatorname {R e L U} \left(W _ {1} f _ {i} \oplus W _ {2} f _ {j}\right)\right) \tag {7}
$$
where  $W_{1}, W_{2} \in \mathbb{R}^{d_{f} \times d_{f}/2}$ ,  $W_{3} \in \mathbb{R}^{d_{f} \times L}$ ,  $L$  is the number of entity types.  $M_{ij} \in \mathbb{R}^{L}$  represents the type probability for a span starts from token  $t_{i}$  and ends at token  $t_{j}$ .
For inner entities, we define the ground truth entity of word pair  $(t_i,t_j)$  as  $\hat{M}_{ij}$ , where  $t_i$  and  $t_j$  are start and end nodes of a span. Cross Entropy (CE) is used to calculate the loss
$$
\begin{array}{l} L _ {i n n e r} = - \left(\sum \left(\hat {M} _ {i j} \log \left(M _ {i j}\right)\right) \cdot I (\mathrm {O}) + \right. \\ \lambda_ {1} \cdot \sum \left(\hat {M} _ {i j} \log \left(M _ {i j}\right)\right) \cdot (1 - I (\mathrm {O})) \tag {8} \\ \end{array}
$$
where  $M_{ij} \in \mathbb{R}^L$  denotes the entity score in the graph module.  $I(\mathrm{O})$  is a switching function to distinguish the loss of non-entity 'O' and other entity types. It is defined as follows.
$$
I (\mathrm {O}) = \left\{ \begin{array}{l} 1, \text {i f t y p e} = ^ {\prime} \mathrm {O} \\ 0, \text {i f t y p e} \neq^ {\prime} \mathrm {O}, \end{array} \right. \tag {9}
$$
$\lambda_{1}$  is the bias weight. The larger  $\lambda_{1}$  is, the greater impacts of entity types, and the smaller influences of non-entity 'O' on the graph module.
2.4 BiFlaG Training
The entity score  $M$  in Eq.(7) carries the type probability of each word pair in the sentence. To further consider the information propagation from inner entities to outer ones, we use Bi-GCN to generate new representations from entity score  $M$  for the flat module. The largest type score  $r_{ij}$  of the word pair  $(t_i, t_j)$  indicates whether this span is an entity or non-entity and the confidence score of being such type, which is obtained by a max-pooling operation:
$$
r _ {i j} = \left\{ \begin{array}{c} \max  \left(m _ {i j}\right), \text {i f t y p e} \neq {} ^ {\prime} \mathrm {O} ^ {\prime} \\ 0, \text {i f t y p e} = {} ^ {\prime} \mathrm {O} ^ {\prime} \end{array} \right. \tag {10}
$$
where type represents the entity type or non-entity 'O' corresponding to the maximum type score. When the corresponding type is  $O$ , there exists no dependencies between  $t_i$  and  $t_j$ , thus we set  $r_{ij}$  to 0. A new graph that carries the boundary information
6411
ACE2005	GENIA
Train	(%)	Dev	(%)	Test	(%)	Train	(%)	Dev	(%)	Test	(%)
# sentences with o.l.	7,285	968	1,058	15,022	1,669	1,854
2,820	(39)	356	(37)	344	(33)	3,432	(23)	384	(23)	467	(25)
# mentions	24,827	3,234	3,028	47,027	4,469	5,596
outermost entity	18,656	(75)	2,501	(77)	2,313	(76)	42,558	(90)	4,030	(90)	4,958	(89)
inner entity	6,171	(25)	733	(23)	715	(24)	4,469	(10)	439	(10)	642	(11)
Table 1: Statistics of the datasets used in our experiments: ACE2005 and KBP2017. o.l.: overlapping mentions.
of inner entities is defined as  $G^3 = (V, E)$ , where  $r_{ij} \in E$ .
The new representation used to update flat module consists of two parts. The first part carries the previous representation of each token
$$
\alpha_ {i} ^ {1} = W _ {r} x _ {i} + b _ {r} \tag {11}
$$
where  $W_{r}\in \mathbb{R}^{d_{x}\times d_{f}}$ ,  $b_{r}\in \mathbb{R}^{d_{f}}$ . The second part aggregates inner entity dependencies of the new graph  $G^3$
$$
\alpha_ {i} ^ {2} = \operatorname {B I - G C N} \left(x _ {i}, G ^ {3}\right) \tag {12}
$$
Finally,  $\alpha_{i}^{1}$  and  $\alpha_{i}^{2}$  are added to obtain the new representation
$$
x _ {i} ^ {\text {n e w}} = \alpha_ {i} ^ {1} + \alpha_ {i} ^ {2} \tag {13}
$$
$x_{i}^{new}$  is fed into the flat module to update the parameters and extract better outermost entities.
For outermost entities, we use the BIOES sequence labeling scheme and adopt CRF to calculate the loss. The losses corresponding to the two representations  $(X$  and  $X^{new})$  are added together as the outermost loss
$$
L _ {o u t e r} = C R F _ {X} + C R F _ {X ^ {n e w}} \tag {14}
$$
Entities in the sequence are divided into two disjoint sets of outermost and inner entities, which are modeled by flat module and graph module, respectively. Entities in each module share the same neural network structure. Between two modules, each entity in the flat module is either an independent node, or interacting with one or more entities in the graph module. Therefore, Our BiFlaG is indeed a bipartite graph. Our complete training procedure for BiFlaG is shown in Algorithm 1.
Our BiFlaG model predicts both outermost and inner entities. The total loss is defined as
$$
L = L _ {\text {o u t e r}} + \lambda_ {2} L _ {\text {i n n e r}} \tag {15}
$$
where  $\lambda_{2}$  is a weight between loss of flat module and graph module. We minimize this total loss during training phase.
3.1 Dataset and Metric
We evaluate our BiFlaG on three standard nested NER datasets: GENIA, ACE2005, and TACKBP2017 (KBP2017) datasets, which contain  $22\%$ ,  $10\%$  and  $19\%$  nested mentions, respectively. Table 1 lists the concerned data statistics.
GENIA dataset (Kim et al., 2003) is based on the GENIAcorpus  $3.02\mathrm{p}^{1}$ . We use the same setup as previous works (Finkel and Manning, 2009; Lu and Roth, 2015; Lin et al., 2019a). This dataset contains 5 entity categories and is split into 8.1:0.9:1 for training, development and test.
$\mathbf{ACE2005^2}$  (Walker et al., 2006) contains 7 fine-grained entity categories. We preprocess the dataset following the same settings of (Lu and Roth, 2015; Wang and Lu, 2018; Katiyar and Cardie, 2018; Lin et al., 2019a) by keeping files from bn, nw and wl, and splitting these files into training, development and test sets by 8:1:1, respectively.
KBP2017 Following (Lin et al., 2019a), we evaluate our model on the 2017 English evaluation dataset (LDC2017E55). The training and development sets contain previous RichERE annotated datasets (LDC2015E29, LDC2015E68, LDC2016E31 and LDC2017E02). The datasets are split into 866/20/167 documents for training, development and test, respectively.
Metric Precision  $(P)$ , recall  $(R)$  and F-score  $(F_{1})$  are used to evaluate the predicted entities. An entity is confirmed correct if it exists in the target labels, regardless of the layer at which the model makes this prediction.
<sup>1</sup>http://www.geniaproject.org/genia-corpus/post-annotation
$^{2}$ https://catalog.ldc.upenn.edu/LDC2006T06 (ACE2005)
6412
Model	ACE2005	GENIA	KBP2017
P	R	F1	P	R	F1	P	R	F1
LSTM-CRF (Lample et al., 2016)	70.3	55.7	62.2	75.2	64.6	69.5	71.5	53.3	61.1
Multi-CRF	69.7	61.3	65.2	73.1	64.9	68.8	69.7	60.8	64.9
layered-CRF (Ju et al., 2018)	74.2	70.3	72.2	78.5	71.3	74.7	-	-	-
LSTM. hyp (Katiyar and Cardie, 2018)	70.6	70.4	70.5	79.8	68.2	73.6	-	-	-
Segm. hyp [POS] (Wang and Lu, 2018)	76.8	72.3	74.5	77.0	73.3	75.1*	79.2	66.5	72.3
Exhaustive (Sohrab and Miwa, 2018)4	-	-	-	73.3	68.3	70.7	-	-	-
Anchor-Region [POS] (Lin et al., 2019a)	76.2	73.6	74.9	75.8	73.9	74.8	77.7	71.8	74.6*
Merge & Label (Fisher and Vlachos, 2019)	75.1	74.1	74.6†	-	-	-	-	-	-
Boundary-aware (Zheng et al., 2019)	-	-	-	75.9	73.6	74.7†	-	-	-
GEANN [Gazetter] (Lin et al., 2019b)	77.1	73.3	75.2*	-	-	-	-	-	-
KBP2017 Overview (Ji et al., 2017)	-	-	-	-	-	-	72.6	73.0	72.8†
BiFlaG	75.0	75.2	75.1	77.4	74.6	76.0	77.1	74.3	75.6
(-0.1*)	(+0.9*)	(+1.0*)
Table 2: Experimental results  ${}^{5}$  on ACE2005,GENIA and KBP2017 datasets. POS and Gazetteer indicates using additional POS tags and gazetteers. † represents previous state-of-the-art results under the same settings with our experiments, * represents state-of-the-art results with POS tags or gazetteers, values in parentheses are also compared with them.
3.2 Parameter Settings
Our model is based on the framework of (Yang and Zhang, 2018). We conduct optimization with the stochastic gradient descent (SGD) and Adam for flat and GCN modules, respectively. For GENIA dataset, we use the same 200-dimension pretrained word embedding as (Ju et al., 2018; Sohrab and Miwa, 2018; Zheng et al., 2019). For ACE2005 and KBP2017 datasets, we use the publicly available pre-trained 100-dimension GloVe (Pennington et al., 2014) embedding. We train the character embedding as in (Xin et al., 2018). The learning rate is set to 0.015 and 0.001 for flat and GCN modules, respectively. We apply dropout to embeddings and the hidden states with a rate of 0.5. The hidden sizes of BiLSTM and GCN are both set to 256. The bias weights  $\lambda_{1}$  and  $\lambda_{2}$  are both set to 1.5.
3.3 Results and Comparisons
Table 2 compares our model to some existing state-of-the-art approaches on the three benchmark datasets. Given only standard training data and publicly available word embeddings, the results in Table 2 show that our model outperforms all these models. Current state-of-the-art results on these datasets are tagged with  $\dagger$  in Table 2, we make improvements of  $0.5 / 1.3 / 2.8F_{1}$  on ACE2005, GENIA, and KBP2017 respectively. KBP2017 contains much more entities than ACE2005 and GE
NIA. The number of entities on test set is four times that of ACE2005. Our model has the most significant improvement on such dataset, proving the effectiveness of our BiFlaG model. More notably, our model without POS tags surpasses the previous models (Wang and Lu, 2018; Lin et al., 2019a), which use POS tags as additional representations on all three datasets. Besides, (Lin et al., 2019b) incorporate gazetteer information on ACE2005 dataset, our model also makes comparable results with theirs. Other works like (Straková et al., 2019)<sup>4</sup>, which train their model on both training and development sets, are thus not comparable to our model directly.
Table 3 makes a detailed comparison on the five categories of GENIA test dataset with a layered model (Ju et al., 2018) and a region-based model (Zheng et al., 2019). Compared with region-based model, layered model seems to have higher precision and lower recall, for they are subject to error propagate, the outer entities will not be identified if the inner ones are missed. Meanwhile, region-based model suffers from low precision, as they may generate a lot of candidate spans. By contrast, our BiFlaG model well coordinates precision and recall. The entity types Protein and DNA have the most nested entities on GENIA dataset, the improvement of our BiFlaG on these two entity types is remarkable, which can be attributed to the in
4Their reported results are 75.36 and 76.44 trained on concatenated train+dev sets on ACE2005 and GENIA, respectively. They also use lemmas and POS tags as additional features.
$^{3}$ Code is available at: https://github.com/cslydia/BiFlaG.  $^{4}$ This result is reported by (Zheng et al., 2019), consistent with our own re-implemented results.
6413
Category	Our model	Boundary-aware	Layered-CRF	Num.
P	R	F	P	R	F	P	R	F
DNA	72.7	72.7	72.7	73.6	67.8	70.6	74.4	69.7	72.0	1,290
RNA	84.4	84.4	84.4	82.2	80.7	81.5	90.3	79.5	84.5	117
Protein	79.5	76.5	78.0	76.7	76.0	76.4	80.5	73.2	76.7	3,108
Cell Line	75.9	67.6	71.5	77.8	65.8	71.3	77.8	65.7	71.2	462
Cell Type	76.7	72.4	74.4	73.9	71.2	72.5	76.4	68.1	72.0	619
Overall	77.4	74.6	76.0	75.8	73.6	74.7	78.5	71.3	74.7	5,596
teraction of nested information between the two subgraph modules of our BiFlaG.
3.4 Analysis of Each Module