This project investigates the application of a Random Forest Classifier for analyzing meta- data from survey papers on large language mod- els (LLMs), a rapidly growing area within AI. The goal is to assist new researchers by pro- viding insights into the trends and patterns in LLM survey publications. Through a structured workflow—comprising data loading, explo- ration, manipulation, and visualization—key attributes such as release dates, categories, and taxonomies were analyzed. Techniques like TF-IDF vectorization, one-hot encoding, and feature scaling were employed to construct a ro- bust feature matrix. Hyperparameter tuning us- ing grid search optimized the classifier’s perfor- mance. Although the model achieved perfect training accuracy, a lower test accuracy (0.39) indicated overfitting, likely caused by dataset imbalance. With a best cross-validation score of 0.26, future improvements will focus on addressing data imbalance, enhancing feature engineering, and exploring alternative models to boost performance. The project highlights trends in LLM research and suggests paths for enhancing model accuracy. 1 Introduction AI techniques have been widely applied to var- ious domains, such as images (He et al., 2016; Dosovitskiy, 2020), texts (Vaswani et al., 2017; De- vlin et al., 2018), and graphs (Kipf and Welling, 2016; Zhuang and Al Hasan, 2022). As a critical subset of AI techniques, Large Language Mod- els (LLMs) have gained significant attention in recent years (Radford et al., 2018, 2019; Brown et al., 2020; Achiam et al., 2023; Bai et al., 2022; Team et al., 2023). Especially, more and more new beginners are interested in the research top- ics about LLMs. To learn the recent progress in this field, new beginners commonly will read sur- vey papers about LLMs. Therefore, to facilitate their learning, numerous survey papers on LLMshave been published in the last two years. How- ever, a large amount of these survey papers can be overwhelming, making it challenging for new beginners to read them efficiently. To embrace this challenge, in this project, we aim to explore and analyze the metadata of LLMs survey papers, pro- viding insights to enhance their accessibility and understanding (Zhuang and Kennington, 2024). Proposed Plan: Specifically, My objective is to systematically ana- lyze the dataset through data loading, exploration, manipulation, and visualization, with the aim of extracting meaningful insights and enhancing both accessibility and understanding. The methodology is divided into the following key phases: Data Loading, Data Exploration, Data Visualization, and Data Validation. Each phase is explained in detail in the later sections of this report. Based on this approach, the timeline for the project is structured as follows: Week 1: Data loading and initial exploration. Week 2: Data manipulation and cleaning. Week 3: Visualization and model validation. Week 4: Finalizing the report in LaTeX. To achieve optimal results, I utilized several li- braries, including Matplotlib and Seaborn, among others, for data visualization and analysis. In short, this proposed plan outlines a systematic approach for analyzing the dataset, allowing for the extraction of meaningful insights and the validation of findings through model testing. Overall, My contributions to the project can be summarized as follows: a) Implementation of a structured data analysis workflow. b) Effective use of visualization libraries for ex- tracting insights. c) Generated bar charts and box plots to effec- tively findings and highlighting patterns in the data. d) Thorough validation of findings through model testing and analysis, then applied the Ran- dom Forest algorithm to evaluate accuracy. 2 Methodology 2.1 Data Exploration This section presents a overview of the key findings from the data exploration process: a) Dataset Overview: The dataset comprises mul- tiple attributes, including ’Release Date’, ’Title’, ’Summary’, ’Categories’, and ’Taxonomy’, which are essential for understanding survey papers. b) Data Types: The ’Release Date’ is formatted as a datetime object, while ’Categories’ and ’Tax- onomy’ are categorical variables, which are crucial for subsequent analyses. c) Missing Values: The dataset was examined for missing values, revealing that all critical columns are complete, ensuring data integrity for analysis. d) Descriptive Statistics: The average number of surveys released each month is approximately 9.6, providing a baseline for understanding publication trends. e) Trends Over Time: By grouping data by year and month, we identified fluctuations in survey releases, highlighting periods of increased or de- creased publication activity. (Fig. 1) /uni00000015/uni00000013/uni00000015/uni00000014/uni00000010/uni0000001a /uni00000015/uni00000013/uni00000015/uni00000015/uni00000010/uni0000001b/uni00000015/uni00000013/uni00000015/uni00000015/uni00000010/uni00000014/uni00000015/uni00000015/uni00000013/uni00000015/uni00000016/uni00000010/uni00000015 /uni00000015/uni00000013/uni00000015/uni00000016/uni00000010/uni00000016 /uni00000015/uni00000013/uni00000015/uni00000016/uni00000010/uni00000017 /uni00000015/uni00000013/uni00000015/uni00000016/uni00000010/uni00000018 /uni00000015/uni00000013/uni00000015/uni00000016/uni00000010/uni00000019 /uni00000015/uni00000013/uni00000015/uni00000016/uni00000010/uni0000001a /uni00000015/uni00000013/uni00000015/uni00000016/uni00000010/uni0000001b /uni00000015/uni00000013/uni00000015/uni00000016/uni00000010/uni0000001c/uni00000015/uni00000013/uni00000015/uni00000016/uni00000010/uni00000014/uni00000013 /uni00000015/uni00000013/uni00000015/uni00000016/uni00000010/uni00000014/uni00000014 /uni00000015/uni00000013/uni00000015/uni00000016/uni00000010/uni00000014/uni00000015/uni00000015/uni00000013/uni00000015/uni00000017/uni00000010/uni00000014/uni00000013/uni00000011/uni00000013/uni00000015/uni00000011/uni00000018/uni00000018/uni00000011/uni00000013/uni0000001a/uni00000011/uni00000018/uni00000014/uni00000013/uni00000011/uni00000013/uni00000014/uni00000015/uni00000011/uni00000018/uni00000014/uni00000018/uni00000011/uni00000013/uni00000014/uni0000001a/uni00000011/uni00000018/uni00000015/uni00000013/uni00000011/uni00000013/uni00000031/uni00000058/uni00000050/uni00000045/uni00000048/uni00000055/uni00000003/uni00000052/uni00000049/uni00000003/uni00000036/uni00000058/uni00000055/uni00000059/uni00000048/uni0000005c/uni00000056/uni00000003/uni00000035/uni00000048/uni0000004f/uni00000048/uni00000044/uni00000056/uni00000048/uni00000047 Figure 1: Trends of survey papers over month f) Taxonomy Distribution: Analysis of the pro- posed taxonomy revealed that 26 survey papers are categorized as "Trustworthy," indicating a signif- icant focus on this area within the dataset. (Fig. 2) g) Visual Insights: Line Plot: Depicted trends in survey releases over time, showcasing publication patterns. (Fig. 1) Bar Chart: Illustrated the distribution of survey papers across various taxonomy categories, clarify- ing research focus areas. (Fig. 2) /uni00000037/uni00000055/uni00000058/uni00000056/uni00000057/uni0000005a/uni00000052/uni00000055/uni00000057/uni0000004b/uni0000005c /uni00000026/uni00000052/uni00000050/uni00000053/uni00000055/uni00000048/uni0000004b/uni00000048/uni00000051/uni00000056/uni0000004c/uni00000059/uni00000048/uni00000033/uni00000055/uni00000052/uni00000050/uni00000053/uni00000057/uni0000004c/uni00000051/uni0000004a/uni00000036/uni00000046/uni0000004c/uni00000048/uni00000051/uni00000046/uni00000048 /uni00000035/uni00000048/uni00000046/uni00000036/uni0000005c/uni00000056/uni00000003/uni00000009/uni00000003/uni0000002c/uni00000035 /uni00000030/uni00000058/uni0000004f/uni00000057/uni0000004c/uni00000010/uni00000050/uni00000052/uni00000047/uni00000044/uni0000004f/uni00000003/uni00000009/uni00000003/uni00000033/uni00000055/uni00000048/uni00000010/uni00000057/uni00000055/uni00000044/uni0000004c/uni00000051/uni0000004c/uni00000051/uni0000004a/uni00000028/uni00000059/uni00000044/uni0000004f/uni00000058/uni00000044/uni00000057/uni0000004c/uni00000052/uni00000051 /uni00000036/uni00000052/uni00000049/uni00000057/uni0000005a/uni00000044/uni00000055/uni00000048/uni00000003/uni00000028/uni00000051/uni0000004a/uni0000004c/uni00000051/uni00000048/uni00000048/uni00000055/uni0000004c/uni00000051/uni0000004a/uni00000024/uni00000047/uni00000044/uni00000053/uni00000057/uni00000044/uni00000057/uni0000004c/uni00000052/uni00000051/uni00000003/uni00000037/uni00000058/uni00000051/uni0000004c/uni00000051/uni0000004a/uni00000035/uni00000052/uni00000045/uni00000052/uni00000057/uni0000004c/uni00000046/uni00000056/uni0000002a/uni00000055/uni00000044/uni00000053/uni0000004b/uni00000056 /uni00000032/uni00000057/uni0000004b/uni00000048/uni00000055/uni00000056/uni0000002f/uni00000044/uni0000005a /uni00000029/uni0000004c/uni00000051/uni00000044/uni00000051/uni00000046/uni00000048/uni00000028/uni00000047/uni00000058/uni00000046/uni00000044/uni00000057/uni0000004c/uni00000052/uni00000051 /uni0000002b/uni00000044/uni00000055/uni00000047/uni0000005a/uni00000044/uni00000055/uni00000048/uni00000003/uni00000024/uni00000055/uni00000046/uni0000004b/uni0000004c/uni00000057/uni00000048/uni00000046/uni00000057/uni00000058/uni00000055/uni00000048 /uni00000037/uni00000044/uni0000005b/uni00000052/uni00000051/uni00000052/uni00000050/uni0000005c/uni00000013/uni00000018/uni00000014/uni00000013/uni00000014/uni00000018/uni00000015/uni00000013/uni00000015/uni00000018/uni00000033/uni00000044/uni00000053/uni00000048/uni00000055/uni00000003/uni00000026/uni00000052/uni00000058/uni00000051/uni00000057Figure 2: Distribution of the proposed taxonomy Box Plot: Analyzed release days, providing in- sights into publication timing relative to different Taxonomy. (Fig. 3) In addition to that I used Box Plot for the above analysis and visualise the dataset. Box plots are considered efficient data visualization tools be- cause they provide a concise summary of a dataset’s distribution in a single, compact graphic. They are particularly useful for comparing multiple datasets or groups and it shows the distribution of release days for each taxonomy. Each box represents a tax- onomy, and the components of the box plot provide the following information: The box plot visualizes the distribution of paper release days across AI subfields. Each box rep- resents the interquartile range (IQR), containing the middle 50% of the data, with the line inside showing the median (50th percentile), or typical release day. The whiskers extend beyond the IQR, showing the overall range of release days, while individual points outside the whiskers represent outliers—unusually early or late releases. Tall boxes, such as in "Multi-modal," indicate greater variability in release times, suggesting pa- pers are published more irregularly. Shorter boxes, like in "Law," imply more consistent timing. Out- liers, particularly in fields like "Pretraining," might indicate groundbreaking papers released outside usual patterns. Comparing median lines across fields (e.g., "Graphs" vs. "RecSys & IR") shows which cat- egories have papers released earlier or later. Skew- ness in the boxes can highlight whether a field tends to have earlier or later paper releases, helping to understand trends and activity timing across AI research. 2.2 Data Manipulation In Data Manipulation outlines the steps taken to create a feature matrix from the dataset: a) Vectorization of Text Data: The ’Title’ and /uni00000026/uni00000052/uni00000050/uni00000053/uni00000055/uni00000048/uni0000004b/uni00000048/uni00000051/uni00000056/uni0000004c/uni00000059/uni00000048/uni00000030/uni00000058/uni0000004f/uni00000057/uni0000004c/uni00000010/uni00000050/uni00000052/uni00000047/uni00000044/uni0000004f/uni00000003/uni00000009/uni00000003/uni00000033/uni00000055/uni00000048/uni00000010/uni00000057/uni00000055/uni00000044/uni0000004c/uni00000051/uni0000004c/uni00000051/uni0000004a /uni00000024/uni00000047/uni00000044/uni00000053/uni00000057/uni00000044/uni00000057/uni0000004c/uni00000052/uni00000051/uni00000003/uni00000037/uni00000058/uni00000051/uni0000004c/uni00000051/uni0000004a /uni00000033/uni00000055/uni00000052/uni00000050/uni00000053/uni00000057/uni0000004c/uni00000051/uni0000004a /uni00000037/uni00000055/uni00000058/uni00000056/uni00000057/uni0000005a/uni00000052/uni00000055/uni00000057/uni0000004b/uni0000005c /uni00000028/uni00000059/uni00000044/uni0000004f/uni00000058/uni00000044/uni00000057/uni0000004c/uni00000052/uni00000051 /uni00000036/uni00000046/uni0000004c/uni00000048/uni00000051/uni00000046/uni00000048 /uni00000035/uni00000048/uni00000046/uni00000036/uni0000005c/uni00000056/uni00000003/uni00000009/uni00000003/uni0000002c/uni00000035 /uni00000035/uni00000052/uni00000045/uni00000052/uni00000057/uni0000004c/uni00000046/uni00000056 /uni0000002a/uni00000055/uni00000044/uni00000053/uni0000004b/uni00000056/uni00000036/uni00000052/uni00000049/uni00000057/uni0000005a/uni00000044/uni00000055/uni00000048/uni00000003/uni00000028/uni00000051/uni0000004a/uni0000004c/uni00000051/uni00000048/uni00000048/uni00000055/uni0000004c/uni00000051/uni0000004a /uni0000002f/uni00000044/uni0000005a /uni00000029/uni0000004c/uni00000051/uni00000044/uni00000051/uni00000046/uni00000048 /uni00000028/uni00000047/uni00000058/uni00000046/uni00000044/uni00000057/uni0000004c/uni00000052/uni00000051/uni0000002b/uni00000044/uni00000055/uni00000047/uni0000005a/uni00000044/uni00000055/uni00000048/uni00000003/uni00000024/uni00000055/uni00000046/uni0000004b/uni0000004c/uni00000057/uni00000048/uni00000046/uni00000057/uni00000058/uni00000055/uni00000048 /uni00000032/uni00000057/uni0000004b/uni00000048/uni00000055/uni00000056 /uni00000037/uni00000044/uni0000005b/uni00000052/uni00000051/uni00000052/uni00000050/uni0000005c/uni00000013/uni00000018/uni00000014/uni00000013/uni00000014/uni00000018/uni00000015/uni00000013/uni00000015/uni00000018/uni00000016/uni00000013/uni00000035/uni00000048/uni0000004f/uni00000048/uni00000044/uni00000056/uni00000048/uni00000003/uni00000027/uni00000044/uni0000005c/uni00000025/uni00000052/uni0000005b/uni00000003/uni00000033/uni0000004f/uni00000052/uni00000057/uni00000003/uni00000052/uni00000049/uni00000003/uni00000035/uni00000048/uni0000004f/uni00000048/uni00000044/uni00000056/uni00000048/uni00000003/uni00000027/uni00000044/uni0000005c/uni00000003/uni00000045/uni0000005c/uni00000003/uni00000037/uni00000044/uni0000005b/uni00000052/uni00000051/uni00000052/uni00000050/uni0000005cFigure 3: Box Plot ’Summary’ columns were converted into numerical representations using TF-IDF vectorization with TfidfVectorizer, creating a sparse matrix that cap- tures the importance of words while excluding com- mon stop words. b) One-Hot Encoding for Categorical Data: The ’Categories’ column was split into separate entries and one-hot encoded using pd.getdummies, trans- forming categorical values into a binary format suitable for modeling. c) Combining Features: A comprehensive fea- ture matrix was constructed by combining the TF- IDF matrices for ’Title’ and ’Summary’ with the one-hot encoded categories. d) Normalization of Features: The feature ma- trix was normalized using MinMaxScaler to scale all features to a range between 0 and 1, which is essential for many machine learning algorithms. e) Label Encoding: The target variable, ’Tax- onomy’, was encoded using LabelEncoder, con- verting categorical labels into numerical values for model training. f) Dataset Splitting: The dataset was split into training and testing sets (70/30) using train and test split, allowing for effective model training and eval- uation. These steps ensured that the raw dataset was effectively transformed into a structured fea- ture matrix, ready for analysis and modeling. 2.3 Data Evaluation I employed the Random Forest Classifier to evalu- ate my dataset. This model is highly appropriate for the task, offering accurate predictions while capturing intricate relationships and emphasizing key features. Its flexibility and robust performance make it especially capable of managing varied and noisy data. Furthermore, I applied grid search for hyperparameter optimization to improve accuracy. For data evaluation, the sklearn library was used totrain the model and test its accuracy. 3 Conclusion In conclusion, I applied a Random Forest Classifier, fine-tuned through hyperparameter optimization, with the following best parameters: Max Depth: 15 Min Samples Leaf: 1 Min Samples Split: 2 Number of Estimators: 200 The model achieved a perfect training accuracy of 1.0 but only 0.39 on the test data, indicating overfitting. This discrepancy is likely due to an imbalanced dataset, primarily composed of 0 and false values. The best cross-validation score was 0.26, suggesting further enhancements are needed. Future work should focus on addressing the dataset imbalance, applying more feature engineering, and considering alternative modeling techniques to im- prove performance. A APPENDIX * This section summarizes the key settings and hyperparameters used for the Random Forest Clas- sifier in this analysis. 1. Model Type: Classifier: Random Forest Classifier 2. Hyperparameters: Number of Estimators: 200 Max Depth: 15 Min Samples Split: 2 Min Samples Leaf: 1 3. Training and Testing Configuration: Data Split: 70% for training and 30% for testing Cross-Validation: 5-fold 4. Data Preprocessing: Vectorization: Applied TF-IDF to the ’Title’ and ’Summary’ Normalization: Used Min-Max scaling on fea- tures Label Encoding: Encoded the target variable ’Taxonomy’ with LabelEncoder These settings were crucial for enhancing the model’s performance and improving the accuracy of predictions.