# Classification and Causes Identification of Chinese Civil Aviation Incident Reports
## 2 Data and Methodology
### Natural Language Processing  
NLP is a popular area in computer science that is closely related to technologies in several fields, including artificial intelligence, Internet technologies, and mathematics. In effect, it enables computers to derive meaningful information from natural language and communicate effectively with humans. NLP utilizes increasing computing power to process a large volume of digital information and has been applied in machine translation, knowledge graphs, and automatic abstract generation [19]. When employing NLP for downstream tasks, the text needs to be pre-processed and vectorized. Standard steps of Chinese text pre-processing include data filtering, spelling standardization, Chinese word segmentation, stop-words removal, and part-of-speech (POS) tagging. Data filtering removes nonstandard reports or reports that are impossible to analyze, which ensures that the data to be analyzed include texts available for research. Spelling standardization automatically corrects misspelled words and replaces abbreviations and synonyms with standard expressions to enhance robustness. Chinese word segmentation divides a sentence into tokens, such as single words and phrases [20]. Conjunctions and adverbs provide almost no valid information for text analysis and thus, are removed to reduce redundancy. POS tagging attaches a tag to each token for its part of speech. However, Chinese does not require the steps of stemming and capitalization normalization usually used in English.  
Text vectorization is an approach that is used extensively to transform unstructured text into a structured representation. Bag-of-words (BoW) representations and word embeddings are the two commonly used models. The BoW representation regards each document as a bag of words, neglecting their order, grammar, and syntax. In effect, each document can be represented as a vocabulary-length vector, in which the values are equal to the number of occurrences of the corresponding words. However, BoW ignores the order of words and cannot reflect their importance. The employment of term frequency-inverse document frequency (TF-IDF) can address this and provides the importance of each word in a document [21], as shown in Equation (1).  
\[x_{ik}=f_{ik}\times log\left(\frac{N}{n_{i}}\right) \tag{1}\]  
where \(f_{ik}\) is the occurrence time of word \(i\) in document \(k\), \(N\) is the number of documents in the dataset, and \(n_{i}\) is the number of documents in which word \(i\) appears. n-grams, also known as combinations of consecutive tokens, can capture the syntactic information of words, and \(n\) denotes the number of tokens. In practice, n is usually 4 or less to avoid overly sparse vectors [22].  
[PERSON] et al. adopted word embedding techniques to derive a dense vector representation of a document [23]. With word embeddings, each word can be represented as a dense vector. Likewise, documents can be represented as a combination of the vectors of their words. The dimensions of the embedded space share potential feature, so models can be trained to capture semantic and syntactic similarities and other linguistic laws. Before solving a specific task, the word embedding model can be trained on an external dataset in the domain to best initialize the vectors of words [23].