Nowadays, there is an influx of researchers emphasizing Large Language Models (LLMs). While the field is broadening, it becomes diffi- cult to keep up all the models, and techniques associated with the novel idea. To tackle this problem, a study has been conducted for assign- ing survey papers to taxonomy in an automated way. In this assignment, I am using their dataset for the task of exploration, manipulation, and evaluation. After finishing the instructed part, I did further exploration by using a cross tab between taxonomy and date, representing dif- ferent visualizations for survey papers by tax- onomy over time, and plotting the box of re- lease day by taxonomy title. The experimen- tal analysis indicates that Logistic Regression (LR) outperformed all the 8 Classifiers in terms of accuracy score, while GaussianNB (GNB) shows the most commendable precision score. For weighted recall and f1 score, LR shows the highest performance in text classification data. 1 Introduction AI techniques have been widely applied to var- ious domains, such as images (He et al., 2016; Dosovitskiy, 2020), texts (Vaswani et al., 2017; De- vlin et al., 2018), and graphs (Kipf and Welling, 2016; Zhuang and Al Hasan, 2022). As a critical subset of AI techniques, Large Language Mod- els (LLMs) have gained significant attention in recent years (Radford et al., 2018, 2019; Brown et al., 2020; Achiam et al., 2023; Bai et al., 2022; Team et al., 2023). Especially, more and more new beginners are interested in the research top- ics about LLMs. To learn the recent progress in this field, new beginners commonly will read sur- vey papers about LLMs. Therefore, to facilitate their learning, numerous survey papers on LLMs have been published in the last two years. How- ever, a large amount of these survey papers can be overwhelming, making it challenging for new beginners to read them efficiently. To embracethis challenge, in this project, we aim to explore and analyze the metadata of LLMs survey papers, providing insights to enhance their accessibility and understanding (Zhuang and Kennington, 2024). Specifically, in this project, I aim to explore, manip- ulate, and evaluate the existing dataset of Zhuang et al (Zhuang and Kennington, 2024)that consists of 144 survey papers for exploring LLMs survey paper with the help of Machine and ensemble learn- ing techniques. I create a bar chart to visualize the survey paper per month in different taxonomies. I want to plot the release day by taxonomy title and figure out the relationship between date and taxonomy by using crosstab analysis. For feeding the dataset into models, I preprocess the dataset so that the machine can learn easily and combine all the features. 8 machine learning algorithms with ensemble methods have been implemented on the dataset for evaluation purposes. I did a detailed analysis by using three figures in the evaluation sec- tion. Overall, My contributions can be summarized as follows: •Analyze the trends of survey papers over time, create a bar chart to see the number of pub- lished papers in different categories, and do a crosstab analysis to understand the relation- ship between taxonomy and date. •Employ vectorizer, normalization techniques, and hot encoding technique to preprocess the data •Several machine learning algorithms along with ensemble techniques have been applied for evaluation 2 Methodology In this section, I will be doing data exploration, data manipulation, and data evaluation. The detailed analysis is discussed below. 2.1 Data Exploration While the quantity of survey papers is massive enough, the instances are small compared to the survey papers. The dataset contains 16 categories of taxonomy along with their title, release date, summary, links, papered, author, and categories. 2021-07 2021-10 2022-01 2022-04 2022-07 2022-10 2023-01 2023-04 2023-07 2023-10 2024-01 Date0.02.55.07.510.012.515.017.520.0Number of Surveys Trends of Survey Papers Over Time Figure 1: Trends of survey paper about large language models over time After loading the dataset by using the pandas library, I try to analyze the trends of the survey paper over time. From Figure 1 we can see, that as time passes, the number of survey papers increases. Trustworthy ComprehensivePromptingScience RecSys & IR Multi-modal & Pre-trainingEvaluation Software EngineeringAdaptation TuningRoboticsGraphs OthersLaw Finance Education Hardware Architecture T axonomy Categories0510152025Number of PapersDistribution of Proposed T axonomy Figure 2: Distribution of survey papers across the tax- onomy Since the release date contains the date, month, and year, I group the number of survey papers by month and year then, count the survey papers by month and convert the data frame for facilitating the plotting of Figure 1. Here For styling the plots matplotlib applies ggplot which provides a profes- sional and cleaner look. The mean value of the survey per month I get is 9.6 after calculating the mean value. Initially, I tried to count the frequency of each category in the taxonomy for analyzing the distri- bution of the proposed taxonomy. I create a bar chart that shows the number of papers is assigned to each category in the taxonomy. The bar chart reveals that most survey papers are published in the 2021-07 2022-08 2022-12 2023-02 2023-03 2023-04 2023-05 2023-06 2023-07 2023-08 2023-09 2023-10 2023-11 2023-12 2024-01 Month0.02.55.07.510.012.515.017.520.0Number of PapersSurvey Papers by T axonomy Over Time T axonomy Adaptation Tuning Comprehensive Education Evaluation Finance Graphs Hardware Architecture Law Multi-modal & Pre-training Others Prompting RecSys & IR Robotics Science Software Engineering TrustworthyFigure 3: Survey paper by taxonomy over time trustworthy category, while fewer are found in fi- nance and hardware architecture. The second most published paper was found in comprehensive and prompting. Software engineering, Multimodal and Pretraining, and Evaluation have a similar number of survey papers.26 survey papers are assigned to the trustworthy taxonomy. 2021-07-28T00:00:00.000000000 2022-12-19T00:00:00.000000000 2023-02-04T00:00:00.000000000 2023-03-03T00:00:00.000000000 2023-03-09T00:00:00.000000000 2023-03-16T00:00:00.000000000 2023-03-22T00:00:00.000000000 2023-04-04T00:00:00.000000000 2023-04-18T00:00:00.000000000 2023-05-10T00:00:00.000000000 2023-05-30T00:00:00.000000000 2023-06-14T00:00:00.000000000 2023-07-05T00:00:00.000000000 2023-07-10T00:00:00.000000000 2023-07-17T00:00:00.000000000 2023-07-24T00:00:00.000000000 2023-08-02T00:00:00.000000000 2023-08-10T00:00:00.000000000 2023-08-15T00:00:00.000000000 2023-08-21T00:00:00.000000000 2023-09-02T00:00:00.000000000 2023-09-13T00:00:00.000000000 2023-09-23T00:00:00.000000000 2023-09-28T00:00:00.000000000 2023-10-08T00:00:00.000000000 2023-10-11T00:00:00.000000000 2023-10-23T00:00:00.000000000 2023-10-27T00:00:00.000000000 2023-11-01T00:00:00.000000000 2023-11-07T00:00:00.000000000 2023-11-13T00:00:00.000000000 2023-11-21T00:00:00.000000000 2023-11-28T00:00:00.000000000 2023-12-05T00:00:00.000000000 2023-12-12T00:00:00.000000000 2023-12-19T00:00:00.000000000 2024-01-01T00:00:00.000000000 2024-01-10T00:00:00.000000000 2024-01-15T00:00:00.000000000 2024-01-23T00:00:00.000000000 2024-01-30T00:00:00.000000000 dateAdaptation Tuning Comprehensive Education Evaluation Finance Graphs Hardware Architecture Law Multi-modal & Pre-training Others Prompting RecSys & IR Robotics Science Software Engineering TrustworthyT axonomy0100000000000000000000000010000000000000000001000101001001000000000010000000000000000000000000000000000000000000000000000 0000000000010000010112010100001000001011100100000000000000000000000000100000000000000000000001001000000000000000000000000 0000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 0000000000000000000000000000000000000100000000000000001000000000000000000000000000010000000000000110100000101000010000000 0000000000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000 0010000000000100000000000000000001100000000000000000100000000000000000000000000000000000000001001001000000000000000000000 0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000 0000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000 0000000010100000000000000000000000010000000000000000000000010000000000000000000000001000010000101000000000010000000000000 0000000000000010000000000000000000000000000010000000000100000000010000000000000000000001000000000000000000000000000000000 1000110100000000000000000000000000000000000001000000000000000000000000001001001110000000010011000000000010000000001100100 0000000000000000000000000000000110001000000000010000010000000100000000000001000000000000000001000000000000010000100000000 0000000000000000000000000000000000000000000000000000000000100000101000000000000000100100001001000000000001000000000000000 0000000000000000001000001001000000000000001000100010000000000000000000000110000000000010100000000000001000001000000000010 0001000000000000000000000000000000000000010000001000000001000000000000010000100000000000000100000000000000100000000010000 0000001001000000100000100000110000000000000000100000000010002111000110000000010001000000100000000000010110000111000001001Crosstab of T axonomy and date 0.000.250.500.751.001.251.501.752.00 Figure 4: Correlation between taxonomy and date Here Figure 4 analyzes the relationship between two categorical variables (taxonomy and release date) using crosstab. The boxplot displays a repre- sentation of the release day by taxonomy title. 2.2 Data Manipulation In this section, I try to manipulate the dataset ac- cording to its nature. Since the dataset contains text values, I need to use a TF-IDF vectorizer for preprocessing. This vectorizer helps to retrieve the information precisely (Abubakar et al., 2022). I vectorize the title and summary column. Later I ap- plied one hot encoding technique in the categories column. This method is useful for converting the categorical data into numerical values to feed the content to the machine since the machine only un- derstands the numerical values (Seger, 2018). Later combining all of the features into a sparse matrix, thus building the feature matrix. Now when all the features are in numerical version, normalization technique has been applied to scale down features range between 0 to 1 and converge the features in a single unit form (Afrin et al., 2022). Identifying Taxonomy as the target column, I try to encode the label. Later, I allocate 40% dataset for testing and the rest 60% for training by setting the test size to 0.4. 2.3 Data Evaluation I have employed 8 machine learning algorithms Support vector machine(Linear, Poly), Gaus- sianNB (GNB), Decision tree (DT), k neighbors classifier (KNN), Logistic regression, XGB clas- sifier, Random Forest (RF) classifier to evaluate the dataset in terms of accuracy, precision, recall, f score. Figure 5: Performance level of all classifiers in terms of accuracy Among all of the classifiers, Logistic regression outperforms in terms of accuracy and achieves 36% by dealing with the sparsity of the data. It works simply with interpretable coefficients indi- cating the intensity of each feature’s influence on the outcome (Ifrim et al., 2008). Gaussian NB is well fit for TF-IDF text presentations by assuming Gaussian distribution for feature independence (Xu, 2018). While RF is powerful and robust, still GNB performs better than RF and shows 34% accuracy. GNB simplifies the calculation making it suitable for text data and leading to better performance. KNN performs worst among all of them by provid- ing 14% accuracy. Text data typically contains a large number of features (for example, words or tokens). KNN is based on distance calculations, which can be less useful in high-dimensional environments due to the "curse of dimensionality." As dimensions expand, data points get sparser, making it more difficult to discover meaningful neighbors (Moldagulova and Sulaiman, 2017). Its inherent limitations make Figure 6: Performance level of all classifiers in terms of precision, recall, f score it less fit for text classification compared to other methods. From Figure 5, we can see SVM lin- ear gives more accuracy (26%) than SVM poly (19%) since text data contains linear separability characteristics. While polynomial kernels may rep- resent noise, linear SVMs can efficiently capture these correlations without adding needless com- plexity. The decision boundaries of linear SVMs are simpler to understand compared to polynomial SVMs (Muflikhah and Haryanto, 2018). While DT provides 24% accuracy, XGBoost achieves 28% in terms of accuracy by emphasizing correcting errors going through sequentially tree construction. It in- tegrates regularization and optimization for better performance than DT. Figure 7: Performance level of all classifiers in terms of precision The above figure demonstrates the weighted pre- cision for each algorithm. While in terms of ac- curacy, LR performs best, GNB outperforms by achieving a 38% precision value. GNB discusses the strength of every independent feature through coefficients and provides impactful outcomes.LR shows 37% preciseness which is second best. XGB and DT achieve respectively 34% and 30% pre- cision. Polynomial SVMs efficiently distinguish classes in datasets where nonlinearity exists among the features by generating complex decision bound- aries. It enhances the precision by allowing the clas- sifier to maintain against errors from non-support vectors. Therefore, polynomial SVM achieves a 36% precision score. 3 Conclusion In this study, several machine learning algorithms along with ensemble learning have been applied to evaluate the performance level in terms of f score, precision, recall, and accuracy level. Data explo- ration has been done to visualize the fluctuation of survey papers over the period and to show the number of papers published in each 16 categories. Most of the papers are published in trustworthy taxonomy. Further, I analyze the correlation be- tween date and taxonomy. The dataset has been processed by using normalization, TF-IDF vector- izer, and encoding techniques. In terms of precision GNB provides the best result while LR performed better in the remaining 3 evaluation metrices. A APPENDIX