# Automatic Gender Identification from Text
## 3 Our Study
### Methodology  
We experimented with eight different classifiers from both traditional machine-learning and deep-learning categories. Applying different classifiers is suitable for a case study on author gender identification from books across different literary epochs because these methods complement each other in capturing diverse textual features. Traditional machine-learning approaches are effective at leveraging handcrafted features, such as stylistic and lexical patterns, which may be indicative of gender within specific epochs. On the other hand, deep learning models, like transformers, excel at automatically learning complex, hierarchical representations from raw text, enabling them to capture subtle contextual and semantic nuances that may transcend specific time periods (epochs). Combining these approaches allows for a comprehensive exploration of the task, accommodating both epoch-specific stylistic differences and broader patterns of gender expression in literary works. This dual methodology ensures robustness and adaptability in dealing with the  
\begin{table}
\begin{tabular}{l c c c} \hline \hline  & **Male** & **Female** & **Total** \\ \hline Training & 5108 & 1320 & 6428 \\ Validation & 715 & 183 & 897 \\ Test & 709 & 188 & 898 \\ \hline Total & 6532 & 1691 & 8223 \\ \hline \hline \end{tabular}
\end{table}
Table 1: The BookSCE, split into training, validation, and testing sets.  
diverse linguistic and cultural variations inherent in the corpus. Below, we elaborate on each category and method employed in our study.  
Traditional ML classifiers:  
Logistic Regression (LR) [22] models with L1 and L2 regularization techniques [23]. We integrated feature selection with chi-squared (\(\chi^{2}\)) [24] statistics and the SelectKBest method into our LR models.  
Extreme Gradient Boosting (XGBoost) [25], which offers robust performance and scalability as an ensemble method. We employed hyperparameter tuning techniques to optimize the performance of the XGBoost model.  
Support Vector Machine (SVM) [26], which is particularly well-suited for binary classification with nonlinear relations.  
Deep Learning classifiers (aka Language Models):  
The Bidirectional Encoder Representations from Transformers (BERT) [14] excels at understanding the context from both directions in a sentence, making it highly effective for tasks like question answering and language inference.  
The Generative Pre-trained Transformer 2 (GPT2) [27] is designed primarily for text generation, performs exceptionally well in creating coherent and contextually relevant text, but its unidirectional approach limits its understanding of full sentence context.  
XLNet [28] integrates the strengths of both BERT and GPT2 by employing a permutation-based training method, allowing it to capture bidirectional context without the limitations of fixed left-to-right training, making it powerful but computationally intensive.  
Robustly optimized BERT approach (RoBERTa) [29] builds on BERT by using more data and longer training periods, resulting in improved performance across various tasks, though at the cost of requiring extensive computational resources.  
Text-to-Text Transfer Transformer (T5) [30] builds on the transformer architecture by reframing all NLP tasks as a text-to-text problem, enabling a unified approach to diverse tasks. It achieves state-of-the-art performance across various benchmarks by utilizing large-scale pretraining and fine-tuning, though it demands significant computational resources for training and inference.  
All these language models leverage pre-trained representations of text data to capture intricate linguistic patterns, providing a robust foundation for gender identification tasks.