# Authorship Detection on Classical Chinese Text Using Deep Learning
## 2 Data and Methods
### Support Vector Machine Based on Term Frequency-Inverse Document Frequency  
SVM technology is a machine learning-based pattern recognition method that has gained considerable attention in the field of artificial intelligence. It is one of the most efficient machine learning models and has become one of the best-performing and most applicable models for pattern recognition. SVM originated from the concept of the optimal separating hyperplane proposed by [PERSON], et al. [28]. It is widely utilized when the training data are clearly labeled, allowing the algorithm to learn from the given data and build a classification model on the extracted features. The dataset is divided into two parts by the model. The linear SVM is employed based on the observation of the results of the experiment.  
Term frequency-inverse document frequency (TF-IDF) is a common weighting technique used in information retrieval and text mining. It is widely applied to evaluate the importance of a word within a document relative to a collection or corpus. The importance of a word increases proportionally with its frequency within the document, but a word's occurrence within a particular document decreases inversely with its frequency across the corpus. The frequency value of the appearance of words in documents is considered term frequency (TF), while inverse document frequency (IDF) refers to the presence of that word throughout the entire text corpus. TF-IDF can be calculated as below  
\[\text{tf}-\text{idf}(\text{word})=\text{tf}(\text{word})\times\text{idf}( \text{word})\text{idf}(\text{word})=\log\frac{\text{n}}{1+\text{df}(\text{ word})} \tag{1}\]  
where TF value is the frequency value of a specific word in a document, n is the total number of documents, and DF is the number of documents that contain that specific word. To prevent division by zero, 1 is added in the denominator.  
In literary novels, every author has a unique writing style, and imitations often reveal traces [29]. In this work we extract linguistic features that reflect each writer's stylistic patterns to serve as training and test data for the SVM model. Chapters 20 through 29 from the first 80 chapters are selected as the training data for the SVM model, as there is no dispute regarding authorship for these. Chapters 110 through 119 from the remaining 40 chapters are selected as the training data. The other chapters are used as test data. Preparation steps included removing redundant spaces and Chinese punctuation marks.  
Function words within each sentence are evenly distributed in classical literary texts [30]. However, differences in each chapter, or document, emerge from the varying frequence of these function words. Our study considers the TF-IDF of function words as the linguistic feature for authorship detection in DRC. A set of 44 function words is chosen, excluding 2 two-letter function words. Assuming that there exists no prior knowledge of differing authorship between the first 80 and last 40 chapters, the model identifies any shift in writing patterns along the chapters. The result of authorship detection is determined by the boundary of that change.  
The process of authorship detection using SVM is summarized as follows:  
1. Pre-process the texts of DRC, dividing it into two parts, each considered manuscripts by two different authors.
2. Divide these two parts into the corresponding chapters and calculate the TF-IDF of function words in chapters as extracted features, forming the whole dataset in two parts.
3. Select ten chapters from both the non-disputed chapters and disputed ones as the training data, with the remaining used as testing data.
4. Train the SVM algorithm on the selected data and build the classification model.
5. Classify the whole dataset and observe the shift in writing patterns, attributing authorship based on the boundary of changing writing habits.