Shifts in StackOverflow : Ask it like Socrates

Community based question-and-answer (Q&A) sites rely on wellposed and appropriately tagged questions. However, most platforms have only limited capabilities to support their users in finding the right tags. In this paper, we propose a temporal recommendation model to support users in tagging new questions and thus improve their acceptance in the community. To underline the necessity of temporal awareness of such a model, we first investigate the changes in tag usage and show different types of collective attention in StackOverflow, a community-driven Q&A website for computer programming topics. Furthermore, we examine the changes over time in the correlation between question terms and topics. Our results show that temporal awareness is indeed important for recommending tags in Q&A communities. 1 Tags in Q&A Communities During the last two decades, various popular question-and-answer (Q&A) platforms, such as Ask.com, Experts-Exchange, Quora, and Yahoo! Answers have emerged. These platforms provide their users with the opportunity to ask questions and challenge other users of the community to share their knowledge based on these questions. Since 2008 StackOverflow (SO) is the de facto standard question-and-answer website for topics in computer science and programming. In SO, users earn reputation through active participation in the community, e.g., by posing well-received questions or providing helpful answers. After achieving a high reputation level, a user is allowed to up-vote or comment on questions and answers or even edit posts of other users. Vasilescu et al. showed that experienced programmers (i.e., active GitHub committers) ask fewer questions and give more answers [9]. Anderson et al. investigated the correlation between delay of and reputation for a given answer [1]. Wang et al. show that the majority of questioners in SO asks only one question during their membership [10]. Only little research has been done on how to support these “newbies” on posing well-received questions. A qualitative study discusses efficient ways on how to give code examples for well-received questions [7]. Bazelli at al. establish a connection between extroversion and openness used in the questions and high reputation points of users [2]. Another means of improving a question is by categorizing the post appropriately and help a skilled responder to find it. To this end SO supports tags to categorize questions. Questioners are forced to choose at least one computer science topic for their new post so that other users, who subscribed topics of their interest and profession, have the chance to find the post. Hence, an appropriately tagged question has a higher probability of getting useful answers. Depending on the social platform, tags are used with different temporal characteristics. In platforms like Twitter many topics, such as sport events or disasters, are discussed in a very narrow time span (e.g., the Superbowl final: #SuperBowl or the Paris attacks of November 2015: #PrayForParis). Lehmann et al. study different classes of collective attention on Twitter [5] and Gruetze et al. evaluate the influence of such temporal changes for the tag recommendation task [4]. Based on the example of Quora, Maity et al. show that the topics discussed in general-purpose Q&A platforms underly strong temporal dynamics (e.g, political topics) [6]. The topics posted on SO (i.e., programming languages, databases, or operating systems) seem to be more static and lead to the assumption that they slowly evolve over time. In contrast to this first intuition, we show that tag usage in SO indeed underlies strong temporal effects. In the following, we discuss four different types of temporal topic popularity patterns in SO. We further show how the likelihood for question terms can strongly change over time for SO topics. Finally, we show that recommender systems that incorporate these temporal changes outperform their static counterparts and thus improve the support for SO users in asking well-received questions to receive appropriate answers and thus earning the Socratic badge, which is awarded for asking a well-received questions. 2 Tags in StackOverflow As the basis for the following analysis, we use the Stack Exchange dataset containing approximately 10 million questions posted between August 1 2008 and July 31 2015 on StackOverflow. The questions were posted by over 4.5 million users and categorized using over 40 thousand unique tags. The dataset is licensed under AttributionShareAlike 3.0 and available online.1 Figure 1b shows the number of questions and answers provided by the SO community. The steady growth of posts emphasizes the increase of popularity and spread of this Q&A platform. We presume that the large 2014 drop in the number of answers and questions originates from the Mighty Mjölnir community update that allows experienced users to close duplicated questions. The temporal dynamics of discussed topics (i.e., applied tags) is shown in Figure 1c. While the number of distinct topics per month increases nearly linearly over time, the number of new topics, i.e., tags that were not used before, does not. During the first two years, the number of new topics is relatively high but strongly decreasing. This shows that the platform is gaining coverage of the relevant topics for developers. As of mid 2010 the number of new topics is relatively stable between 200 and 400 per month. We believe that these topics are real new topics, i.e., topics that emerge at this point in time, such as a new programming languages (e.g. Dart (2011)) or new frameworks (e.g. Apache Spark (2013)). The continuing increase of distinct tags per month originates from a gain in topic coverage and is asymptotic to the total number of topics in SO. Figure 1d shows the stability of the most questioned topics, i.e., the top 100, 250, and 500 tags. The top-100 topics are very stable such that, starting from 2010, these 1 https://archive.org/details/stackexchange Number of . Value Questions 9,970,064 Answers 16,502,856 Users 4,551,130 Tags 29,497,960 Unique Tags 41,719 (a) Dataset statistics 0 100,000 200,000 300,000 400,000 '09 '10 '11 '12 '13 '14 '15 Years N um be r of P os ts Questions Answers (b) Monthly questions & answers 0 10,000 20,000 30,000 '09 '10 '11 '12 '13 '14 '15 Years N um be r of U ni qu e Ta gs New Tags Used Tags (c) Monthly tag usage 0 100 200 300 400 500 '09 '10 '11 '12 '13 '14 '15 Years O ve rla p of to p− k Ta gs