Automatic Classification of Students on T witter Using Simple Profile Information (2020)

Abstract: Obtaining social media demographic information using machine learning is important for efficient computational social science research. Automatic age classification has been accomplished with relative success and allows for the study of youth populations, but student classification—determining which users are currently attending an academic institution—has not been thoroughly studied. Previous work (He et al., 2016) proposes a model which utilizes 3 tweet-content features to classify users as students or non-students. This model achieves an accuracy of 84%, but is restrictive and time intensive because it requires accessing and processing many user tweets. In this study, we propose classification models which use 7 numerical features and 10 text-based features drawn from simple profile information. These profile-based features allow for faster, more accessible data collection and enable the classification of users without needing access to their tweets. Compared to previous models, our models identify students with greater accuracy; our best model obtains an accuracy of 88.1% and an F1 score of .704. This improved student identification tool has the potential to facilitate research on topics ranging from professional networking to the impact of education on Twitter behaviors.

## introduction in recent years, social media has become the focus of more scientific research studies @xcite , and is being used as a valuable means of understanding human tendencies and social circles. due to rapid developments in technology, social media has become a hub for scientific communication and research @xcite , a source of entertainment and news, and a place for networking and socialization (whiting and williams, * authors contributed equally 2013). twitter is one of the most popular social media platforms, especially among young people; 44% of young adults aged 18-24 and 31% of people aged 25-29 use twitter, with 25% using it daily and 14% using it several times per day. 1 the ability to group twitter users by demographic information is crucial in understanding the behaviors of certain subsets of the population. twitter provides limited information on user demographics, necessitating the development of machine learning models to predict non-explicit user features. several studies have demonstrated that it is possible to classify users by gender @xcite , age @xcite @xcite , whether or not they are an organization @xcite , and more. in this paper, we aim to classify twitter users as students or non-students. while many age classification models have been proposed, understanding this subset of the age classification problem may provide new insights into behavior on twitter. grouping users into students and non-students has applications ranging from professional networking @xcite to understanding these users' role in the spread of misinformation, and by extension, the role that education plays in their online behaviors @xcite . previous studies of students' social media interactions have obtained student samples using relatively low-yield, lowaccuracy, or low-coverage techniques such as surveying individuals, analyzing geographic proximity to educational institutions, or manually annotating users one by one @xcite @xcite @xcite . research into automatic demographic prediction of student users could facilitate such student-based studies. the most prominent application of an automatic classification model to student demographic research is described in @xcite . this study develops a machine learning model that uses user tweet content to identify students on twitter and match them with professional mentors on linkedin. despite the model's relatively high accuracy (84%), its use of user tweet content is restrictive because it is not always possible to obtain complete tweet data. protected accounts cannot be classified using this model, and researchers without advanced twitter api access may not be able to obtain sufficient tweet data. here, we propose new models for student classification on twitter which achieve high accuracy using only simple profile information. our use of profile-based features obviates the need for tweet content, which allows for the classification of protected accounts and makes our model more accessible to researchers without access to advanced twitter apis. in addition, user profile information can be requested from the twitter api at a faster rate than a sample of user tweets. these novel student identification models thus show potential for more accurate, accessible, and efficient classifications in demographic studies of social media users. 2 ## data creating the dataset used to train the student identification model required obtaining a set of twitter users who could be labeled as student or non-student and then manually labeling each user. these users were obtained from a combination of a general sample of twitter users, a sample of the followers of a university twitter account, and twitter's "who to follow" feature. ultimately, 1,037 users were labeled, with 225 students and 812 nonstudents. the distribution of the sample is shown in a user was labeled as a non-student if their online profile showed that they had graduated from a higher institution, listed a full-time job, or were organizations or spam bots. general examples included noting a past graduation date or degree in their twitter bio or listing jobs or graduation dates on an associated linkedin profile. labeling was conducted by the listed authors, and inter-annotator agreement was measured on a representative subset of 435 users. the subset demonstrated substantial agreement as shown by a cohen's kappa of 0.73 @xcite . in the case of a disagreement, the first ascribed label was used in the final dataset. to avoid annotator bias in assigning these labels, each annotator labeled half of the dataset then reviewed the other annotator's labels blind. ## results before considering gray area, the stacked classifier outperformed all other classifiers in accuracy, f1, and auroc. every model outperformed the "if-statement classifier" in both accuracy and f1 score, indicating that the additional complexity and extra features considered in the machine learning models contributed to their success. ## conclusions in this paper, we introduce a metadata-based machine learning model to accurately predict student twitter users. we also introduce a gray-area model that achieves 90.3% accuracy without leaving many users unlabeled. our models improve upon past research by providing more accurate, more efficient, and faster classifications due to their use of only simple profile information. currently, we are working to apply this student classifier in a preliminary study of student interactions with covid-19 related misinformation on twitter.