MiQs : Characterization and Prediction of Migrated Questions on StackExchange

StackExchange (SE) is a network of very popular Community based Question and Answer (CQA) websites. Each SE Q&A website is created to address questions on specific user interest or domain. However, several times it is noticed that user post questions on SE websites which do not match the domain of website. Such questions are considered as off-topic for the origin site. Off-topic questions need to be detected and migrated to more appropriate (on-topic) site in SE network. Off-topic questions are migrated manually to other sites by moderators or experts users (through voting process). The process of migrating questions from one site to other is known as Question Migration. Manual question migration is undesirable as it puts extra load on moderators or expert users. It can also potentially cause delay to the user seeking answer of his questions and can also lower down satisfaction level of the user. This work presents first study of migrated questions on SE Q&A website. We present results of characterization study performed on five popular SE Q&A sites: Superuser, Serverfault, StackOverflow, dba.StackExchange, and programmers.StackExchange. We use publically available SE Q&A sites dataset (span of 4.7 years). We perform an in-depth characterization study on 38609 migrated questions. We present results of the temporal distribution, community structure of Q&A websites, owner reputation, amount of discussion, and popularity of migrated questions. We identify several distinguishing features of migrated questions and propose a machine learning based framework to predict migrating questions. Effectiveness of proposed model is tested on five SE Q&A sites. Result shows that proposed model is effective (shows maximum accuracy of 73%) in predicting migrating questions. I. RESEARCH MOTIVATION AND AIM StackExchange (SE) is a network of very popular Community based Question and Answer (CQA) websites1. SE network consists of 112 Q&A sites, 4.6 million users, 8 million questions and 14.2 million answers (on Jan-2014). All SE Q&A website are created to address questions of specific domain or user interest. For example StackOverflow2 is created for addressing programing questions, english.stackexchange3 is created for english language enthusiasts, travel.stackexchange4 is created for travel related questions. Each Q&A website explicitly mentions its domain & guidelines for posting questions. However, several times when user posts a question on a website it does not matches the domain of that website. 1http://stackexchange.com/ 2http://stackoverflow.com/ 3http://english.stackexchange.com/ 4http://travel.stackexchange.com/ Such questions are considered as ‘off-topic’ questions. ‘Offtopic’ questions needs to be detected and migrated to other more appropriate (‘on-topic’) website in SE network. Figure 1 shows snapshot of a migrated question5. Question present in Figure 1 is asked on StackOverflow but later on migrated to meta.StackOverflow. StackOverflow is a Q&A website for programing enthusiasts and hence expects questions related to programing. However this question is not related to programing, in this question user is asking some issue related to posting comments on StackOverflow website i.e issue related to a feature of the website. This questions does not match the interest domain of StackOverflow and hence migrated to meta.stackOverflow. meta.stackOverflow addresses meta questions related to SE Q&A websites and hence a perfect match for this question. Fig. 1: Shows snapshot of a question migrated from StackOverflow to meta.StackOverflow Identifying and migrating off-topic questions is an important activity for SE Q&A websites. SE websites does not only provide answer(s) to the question owner but also serves as knowledge base for the future users [1]. Many times it is noticed that an off-topic question on one website is an exact duplicate of an on-topic question on some other website in SE network [2]. Not-migrating or ignoring such off-topic questions will cause scattering of information. This may later on can create confusion and can potentially make it harder for future users to find similar information. Hence identifying and migrating appropriate questions is important to maintain content quality of SE websites. Currently questions on SE websites are migrated manually by moderators and expert users. Manual migration is tedious 5http://meta.stackoverflow.com/questions/143306/newline-instackoverflow-comments process (see section III) and puts extra load on moderators and expert users. Also manual migration can only be done after posting the question i.e. after some users have spent their time and effort in reading and answering the question. In this work, our aim is to analyze migrated questions and to create an automated technique for questions migration (at the time of post creation). Automated techniques for predicting migrating questions at the time of post creation will not only reduce the load of moderators or expert users but will have multiple other benefits, some of them are listed below: • Improving Response Time: Manual migration can be time consuming and user may not get appropriate answer/response to the question until it is migrated, and hence automated migration can help in improving response time to the user. • Reputation Loss Issue: Question migration causes reversal of all the reputations earned (earned on origin site before migration) by contributing to a migrated question. This causes reputation loss to the people who have earned reputation by answering or commenting on migrated question (on origin site) [3]. This can create dissatisfaction among the user community 6. • Improving Question Quality: If a user gets an early feedback about his off-topic question, he may improve the question content to make it more clear. This process can help in improving the question quality, which over the period of time will improve quality of the Q&A website. The broad research aim of the work presented in this paper is to analyze migrated questions on SE Q&A sites and to reduce load of moderator and expert users by providing framework for automated questions migration. The specific research aim of the work presented in this paper is following: 1) To conduct an in-depth characterization study of migrated questions on SE Q&A sites. To investigate temporal distribution of migrated questions on SE websites and to identify community structure of Q&A website with respect to mutual question migration. To identify properties of migrated questions (such as reputation of owner, amount of discussion, popularity etc.). 2) To identify distinguishing features of migrated questions and to build a predictive model to identify migrating questions from Q&A sites (at the time of post creation). II. RELATED WORK AND RESEARCH CONTRIBUTION CQA sites are very helpful for developers. A recent study done by Gomez et al. [4] shows that StackOverflow helps in spreading innovations in programing as often developers share links related to API’s, tools and libraries [4]. Barua et al. [5] proposed an LDA based model to identify current trend of various programing tools such as languages, API’s, source code management systems etc. Lower quality questions on StackOverflow often receive lower quality answers [6] and 6http://meta.stackoverflow.com/questions/78452/migrated-posts-losetheir-pre-migration-reputation hence to maintain quality of Q&A website it is necessary to remove lower quality questions. Lezina et al. [7] and Correa et al. [8] work on predicting ‘closed’ questions on StackOverflow. Lezina et al. and Correa et al. identified various features related to user-profile, post-feature, tags etc. and proposed machine learning based framework for predicting closed questions. Sometimes many low quality and high quality answers are available for a given question. Identifying best answer among them can be challenging for website community and future users. Tian et al. [9] work on identifying best answer from the given set of answer to a questions and reported accuracy of 72%. On StackOverflow it is mandatory to assign appropriate tags to the question before posting it on the website. However assiging suitable tags is challenging for new users. Xia et al. and Lipczak et al. work on predicting tags for Q&A sites and social networking sites [10], [11]. Xia et al. proposed ‘TagCombine’ model for tag recommendation on software information sites. Lipczak et al. proposed a feedback based tag recommendation method [10]. Lipczak et al. worked on multiple aspects (generality, efficiency and adaptability) of tag recommendation problem. Expert users are backbone of any CQA website. Identifying expert users at an early stage can be beneficial for the website community. Pal et al. and Attias el al. work on identifying experts on CQA sites [12], [13]. Gaussian model proposed by Pal et al. shows recall of 92% [12] as comoared to activity pattern based model of Attias et al. which shows precision of 70%. In another work, Pal et al. [14] identified different classes of experts: C (consistently active), E (active initially), and L (active later) and proposed an SVM based model and reported precision of 94% on StackOverflow. A. Research Contributions In this paper, we work on analyzing and predicting migrated questions on SE Q&A sites. To the best of our knowledge this is the first study of migrated questions on SE Q&A sites, following are the specific and novel research contribution of this work: • An in-depth characterizing study of migrated questions on five popular SE Q&A websites. We identify several research questions and conduct various experiment (on dataset of ≈38,000 migrated questions) to answer those questions (refer Section V). • Machine learning based framework for predicting migrating questions at the time of post creation. We test the effectiveness of proposed framework on five popular SE Q&A websites. Proposed framework is found to be effective with maxi